Should this systematic review and meta-analysis change my practice? Part 2: exploring the role of the comparator, diversity, risk of bias and confidence

Correspondence to Dr Mervyn J Travers, School of Physiotherapy, University of Notre Dame Australia, Fremantle, WA 6160, Australia; [email protected]

Introduction

Systematic review and meta-analyses (SRMA) represent the highest quality of evidence of the effects of interventions and have the capacity to usefully inform clinical decisions, reduce research waste and direct policy making.^1–4 Interventional SRMA often amalgamate the results of multiple randomised controlled trials (RCTs) to determine the magnitude of the estimate of the effect for a chosen intervention and to examine variation among study outcomes (heterogeneity).⁵ As such they represent the highest level of evidence to inform clinicians on the usefulness of a given treatment. Of course, the robustness of SRMA findings is dependent on the quality of the trials that have been pooled. The aim of this two-part Educational Editorial Series is to outline a process of scrutiny and analysis for SRMA to facilitate answering the important question ‘Should this SRMA change my practice?’. An understanding of key features of SRMA which have been outlined in related BJSM Educational Editorials is needed by clinicians in order to make this decision.^{2 6–10} Here, we present a worked example for clinicians illustrating some important considerations when reading an interventional SRMA. Specifically, we highlight the need to consider the pooling of comparators, clinical diversity, comparisons to active control, risk of bias and confidence in the results when interpreting interventional SRMA findings.

The example SRMA

The example SRMA investigated the efficacy of platelet-rich plasma (PRP) injections for tendinopathy.¹¹ Based on data extracted from 16 RCTs (see online supplementary file 1 for a list of references), the authors reported a moderate treatment effect in favour of PRP with a standard mean difference (SMD) of 0.47 (95% CI 0.22 to 0.72, p<0.001). Part 1 of this Educational Editorial series demonstrated issues of methodological reporting, transparency and reproducibility which undermined the trustworthiness of this result and we presented a revised estimate of treatment effect of PRP for tendinopathy (figure 1). Our result was derived from an incomplete dataset. Nonetheless, it suggested a moderate treatment effect in favour of PRP injections similar to the original authors’. Next, we re-examine this result considering the choice of comparator(s).

Compared to what? Considering the role of the control intervention

The...