Identifying the ‘incredible’! Part 1: assessing the risk of bias in outcomes included in systematic reviews

Correspondence to Fionn Büttner, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin 4, Ireland; [email protected]

Introduction

Systematic reviews fulfil a vital role in modern medicine.¹ However, the results of systematic reviews are only as valid as the studies they include.² Pooling flawed, or biased, results from different studies can compromise the credibility of systematic review findings. Bias is a systematic deviation from the truth in the results of a research study that can manifest due to limitations in study design, conduct, or analysis.³

The results of sport and exercise medicine research, like results in other fields, are vulnerable to bias.⁴ It is important that systematic review authors assess for bias in a way that enables a judgement about whether a review outcome is at risk of bias due to methodological limitations in included studies. This two-part education primer focuses on how systematic review authors can perform and interpret risk of bias assessments to avoid misleading systematic review conclusions. In this editorial, we introduce the concept of risk of bias, and the principles of assessing risk of bias.

Bias: the basics

Different biases have effects that vary in direction and magnitude.^{3 5} It is challenging to precisely determine how bias may overestimate or underestimate a study’s true findings. In fact, bias does not always result in distorted study findings and one can never be certain that bias is present when a study has methodological limitations. However, methodological limitations in study design, conduct, or analysis can be consistently associated with inflated research findings.⁵ Due to this uncertainty, study outcomes are considered to be at risk of bias rather than ‘biased’.

Studies with ‘some concerns’ or at ‘high’ risk of bias in design, conduct, analysis, or reporting are at greater risk of inflated findings compared with studies at ‘low’ risk of bias, negatively affecting the probability that study findings accurately reflect reality.^{5 6} Assessing the risk of bias of study outcomes that are included in a systematic review allows readers to interpret the credibility of review findings.

Do not confuse risk of bias with study quality

Risk of bias is a clearly defined term and refers to the perceived risk that the results of a research study...