Effect size quantification for interrupted time series analysis: implementation in R and analysis for Covid-19 research

Abstract

Background

Interrupted time series (ITS) analysis is a time series regression model that aims to evaluate the effect of an intervention on an outcome of interest. ITS analysis is a quasi-experimental study design instrumental in situations where natural experiments occur, gaining popularity, particularly due to the Covid-19 pandemic. However, challenges, including the lack of a control group, have impeded the quantification of the effect size in ITS. The current paper proposes a method and develops a user-friendly R package to quantify the effect size of an ITS regression model for continuous and count outcomes, with or without seasonal adjustment.

Results

The effect size presented in this work, together with its corresponding 95% confidence interval (CI) and P-value, is based on the ITS model-based fitted values and the predicted counterfactual (the exposed period had the intervention not occurred) values. A user-friendly R package to fit an ITS and estimate the effect size was developed and accompanies this paper. To illustrate, we implemented a nation population-based ITS study from January 2001 to May 2021 covering the all-cause mortality of Israel (n = 9,350 thousand) to quantify the effect size of Covid-19 exposure on mortality rates. In the period unexposed to the Covid-19 pandemic, the mortality rate decreased over time and was expected to continue decreasing had Covid-19 not occurred. In contrast, the period exposed to the Covid-19 pandemic was associated with an increased all-cause mortality rate (relative risk = 1.11, 95% CI = 1.04, 1.18, P < 0.001).

Conclusion

For the first time, the effect size in ITS: was quantified, can be estimated by end-users with an R package we developed, and was demonstrated with data showing an increase in mortality following the Covid-19 pandemic. ITS effect size reporting can assist public health policy makers in assessing the magnitude of the entire intervention effect using a single, readily understood measure.