Random error units, extension of a novel method to express random error in epidemiological studies

Abstract

Currently used methods to express random error are often misinterpreted and consequently misused by biomedical researchers. Previously we proposed a simple approach to quantify the amount of random error in epidemiological studies using OR for binary exposures. Expressing random error with the number of random error units (REU) does not require solid background in statistics for a proper interpretation and cannot be misused for making oversimplistic interpretations relying on statistical significance. We now expand the use of REU to the most common measures of associations in epidemiology and to continuous variables, and we have developed a Stata program, which greatly facilitates the calculation of REU.

Acknowledgment

The authors wish to thank Dr Kenneth J. Rothman (Research Triangle Institute and Boston University School of Public Health) for the important discussion and advices that initiated this work.

Disclosure

The authors report no conflicts of interest in this work.

Supplementary materials

Part I. Calculation of random error units for different measures of effect using our postestimation command in Stata

The reu command is available for download from the Boston College Archive. To install it, type at command line: ssc install reu.

1. OR . // Example for calculating the number of random error units for an OR . glm outcome exposure covar_1...covar_n, fam(bin) link(logit)

. reu exposure //exposure is either binary or continuous; for dummies, each dummy needs be mentioned after the reu command // works equally well with logistic or logit procedures 2. Incidence rate ratio . // Example for calculating the number of random error units for an incidence rate ratio . glm outcome exposure covar_1...covar_n, fam(possion) link(log)

. reu exposure //exposure is either binary or continuous, for dummies each dummy needs be mentioned after the reu command // works equally well with Poisson procedure 3. HR . // Example for calculating the number of random error units for HR . stset time, failure(outcome)

. stcox exposure covar_1...covar_n //exposure is either binary or continuous; for dummies, each dummy needs be mentioned after the reu command . reu exposure 4. Risk ratio . // Example for calculating the number of random error units for risk ratio . binreg outcome exposure covar_1...covar_n, rr //exposure is either binary or continuous; for dummies, each dummy needs be mentioned after the reu command . reu exposure 5. Risk difference . // Example for calculating the number of random error units for risk difference . binreg outcome exposure covar_1...covar_n, rd //exposure is either binary or continuous; for dummies, each dummy needs be mentioned after the reu command . reu exposure // works equally well with linear regression Part II. Derivation of the method to calculate REU as presented in Table 1

1. OR:

SE of log OR = √(1/a+1/b+1/c+1/d)

where a, b, c, d refer to those having both the outcome and the exposure, those not having the outcome but being exposed, those having the outcome but not being exposed, and those without the outcome nor exposure, respectively.

Since in the gold standard a=b=c=d=250,000, it follows that SE in the gold standard is 0.004. 2. Incidence rate ratio/HR SE of log incidence rate ratio=√(1/a+1/b)

where a and b refer to exposed and unexposed cases, respectively.

Since a=b=250,000, it follows that SE in the gold standard is 0.0028284. 3. Risk ratio SE of log risk ratio=√(1/a+1/b–1/c–1/d)

where a, b, c, d refer to exposed and unexposed cases, total number of exposed, and unexposed individuals, respectively.

Since a=b=250,000 and c=d=500,000, it follows that SE in the gold standard is 0.002. 4. Risk difference SE of risk difference=√(a(c–a)/c³+b(d–a)/d³)

where a, b, c, d refer to exposed and unexposed cases, total number of exposed, and unexposed individuals, respectively.

Since a=b=50 and c=d=500,000, it follows that SE in the gold standard is 0.00002.

Part III. Demonstration of the interpretation of the REU

The number of random error units shows how many times more individuals an actual study would need to achieve the precision of the gold standard study. First we start the demonstration of this interpretation with an example for the OR. We consider a study on 100 individuals, half of them exposed to a dichotomous exposure that has no effect on the – likewise dichotomous – outcome, which is also present in half of the individuals. The standard error of the log OR in this study is 0.4, and consequently the number of random error units is 10,000. If we multiply this study with 10,000 (keeping the proportion of exposed and those with an outcome constant), we are getting exactly the proposed gold standard study (ie, a study on one million individuals, half of them exposed to a dichotomous exposure that has no effect on the outcome, which is also present in half of the individuals). More generally, decreasing the standard error of a study by a factor of n requires n² times as many observations (providing that the distribution of the exposure and outcome is constant).

SE/n=1/n√(1/a+1/b+1/c+1/d)= √(1/(n²a)+1/(n²b)+1/(n²c)+1/(n²d))

The same can be shown for SE for the rest of the measures of associations.

References

1. Rothman KJ. Curbing type I and type II errors. Eur J Epidemiol. 2010;25(4):223–224.

2. Stang A, Poole C, Kuss O. The ongoing tyranny of statistical significance testing in biomedical research. Eur J Epidemiol. 2010;25(4):225–230.

3. Goodman S. A dirty dozen: twelve p-value misconceptions. Semin Hematol. 2008;45(3):135–140.

4. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337–350.

5. Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process and purpose. Am Stat. 2016;70(2):129–133.

6. Stang A, Deckert M, Poole C, Rothman KJ. Statistical inference in abstracts of major medical and epidemiology journals 1975-2014: a systematic review. Eur J Epidemiol. 2017;32(1):21–29.

7. Fidler F, Thomason N, Cumming G, Finch S, Leeman J. Editors can lead researchers to confidence intervals, but can’t make them think: statistical reform lessons from medicine. Psychol Sci. 2004;15(2):119–126.

8. Mccormack J, Vandermeer B, Allan GM. How confidence intervals become confusion intervals. BMC Med Res Methodol. 2013;13:134.

9. Trafimow D, Marks M. Editorial. Basic Appl Soc Psych. 2015;37(1):1–2.

10. Janszky I, Bjørngaard JH, Romundstad P, Vatten L. A novel approach to quantify random error explicitly in epidemiological studies. Eur J Epidemiol. 2011;26(12):899–902.