Content area
Full text
REANALYZING A META-ANALYSIS ON EXTRASENSORY PERCEPTION DATING FROM 1940, THE FIRST COMPREHENSIVE META-ANALYSIS IN THE HISTORY OF SCIENCE
HOLGER BÖSCH
ABSTRACT: The book Extra-Sensory Perception After Sixty Years (ESP-60), published in 1940, is the first meta-analysis in book format. ESP-60 provides a comprehensive and balanced review of 145 reports on ESP experiments published from 1882 to 1939. The authors discuss the quality and reliability of the experimental data and provide an in-depth discussion of arguments questioning the data altogether. They are the first meta-analysts to take into account publication bias and calculate a "fail-safe N" in order to assure the reliability of their analyses long before such sensitivity analyses became standard practice in meta-analysis. All statistical analyses provided in ESP-60 are based on z-score statistics. The purpose of this paper is to reanalyze the data using an effect-size based approach. Altogether 139 independent effect-sizes measures could be calculated. The distribution of effect-sizes dramatically deviates from the expected funnel shape around the overall mean pi = .57. A striking small-study effect was found. An analysis of samplesize quartiles illustrates the highly visible connection between effect size and sample size. The smallest studies (Q^sub 1^) in the sample (M = 36 trials) have an effect-size of pi = .99, indicating that on average the hit probability of these trials is 99% when MCE is 50%. The largest studies (Q^sub 4^) in the sample (M = 113,716 trials) have an effect-size of pi = .52, indicating that at 52% the hit probability of these trials is just above MCE. An explanation for the dramatic small-study effect cannot be directly derived from the data. Several factors besides publication bias, such as true heterogeneity, data irregularities, and chance, must be considered. However, these data are especially susceptible to publication bias because the early experimental approach to ESP evolved from case studies that principally aimed at demonstrating ESP. Although the authors of ESP-60 have done everything to ensure the completeness of their data, they have underestimated the problem of publication bias. That studies are published independently of their p-value is an ideal that, so far, no empirical science has accomplished. Research in ESP is especially affected because ESP studies are generally not very cost and time intensive, factors that are clearly...