A methodological survey on reporting of pilot and feasibility trials for physiotherapy interventions: a study protocol

Abstract

Introduction

Pilot and feasibility trials aim to test whether a full trial can be conducted or if any procedures must be changed for the full trial. Pilot trials must be reported in a transparent, accurate and complete way. In this report, we present a protocol for a methodological survey with the following aims: (1) to determine the percentage of physiotherapy trial reports which claim to be pilot or feasibility trials that evaluate feasibility, (2) to determine the aspect of feasibility evaluated in the primary objectives of the pilot or feasibility trials, (3) to describe the completeness of reporting of abstracts and full articles of pilot or feasibility trials using the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials and (4) to investigate factors associated with completeness of reporting of pilot or feasibility trials.

Methods and analysis

Reports of randomised controlled trials indexed in the Physiotherapy Evidence Database (PEDro) that claim to be pilot or feasibility trials and published in 2011–2017 will be included. Two independent reviewers will confirm eligibility and classify the aspect of feasibility being evaluated in the objectives of the included pilot or feasibility trials. Completeness of reporting of both the abstract and the full article will be evaluated using the CONSORT extension to randomised pilot and feasibility trials. The primary analysis will be a descriptive analysis about the reporting quality of abstracts and full texts of pilot and feasibility trials. We will use generalised estimating equation analysis to explore factors associated with completeness of reporting.

Ethics and dissemination

The results of this study will be disseminated by presentation at conferences and will be submitted for publication in a peer-reviewed journal. Ethical approval is not necessary for this study.

Correspondence to Dr Lucíola da Cunha Menezes Costa; [email protected]

Strengths and limitations of this study

• This study will be the first to evaluate the quality of reporting of abstracts and full articles of pilot or feasibility trials in the field of physiotherapy using the Consolidated Standards of Reporting Trials statement extension for pilot and feasibility studies.

• All data will be extracted by two independent reviewers in order to increase precision.

• Findings from this study are restricted to pilot and feasibility trials published between 2011 and 2017 indexed in Physiotherapy Evidence Database. Therefore, the results of this study cannot be generalised to all existing pilot and feasibility trials in physiotherapy.

Introduction

Pilot and feasibility trials are exploratory studies that aim to investigate whether the crucial components of planning or preparing for a larger and definitive randomised controlled trial will be viable.^{1 2} They are intended to provide useful information with regard to planning complex interventions (eg, identifying the optimal dose and testing safety), testing study procedures (eg, the form of randomisation, estimation of the recruitment rate and the plausibility of multicentre collaborations), investigating surrogate outcomes or estimating parameters to help perform sample size calculation.^{3 4} Although pilot and feasibility trials have slightly different definitions, both are designed to establish whether the main or definitive trial can and should be conducted in the future, and, if so, to determine how the main trial should be done.⁵ Pilot and feasibility trials are designed to ensure that the main trial will be achievable, rigorous and economically justifiable in order to avoid waste of resources.⁴ However, without a clear understanding of how the pilot or feasibility trial was conducted, researchers and clinicians would not be able to judge the methodological quality and to clinically appraise the published report of the trial.

Evaluations of published pilot and feasibility trials suggests that the trials may not actually be evaluating feasibility^{6 7} and are being poorly reported.⁷ In a small sample of 93 pilot and feasibility trials published in Indian biomedical journals, 68% of the trials performed between-group statistical comparisons and none reported feasibility objectives.⁷ In addition, an ad hoc list of trial characteristics was used to evaluate reporting, rather than a scale or a checklist. Another survey of 191 pilot and feasibility trials published in 1987–2015 in a single journal (ie, Clinical Rehabilitation)⁶ revealed that 110 (58%) trials actually tested feasibility for a future trial, with only 23 trials being followed by a definitive trial.⁶ This implies that the terms ‘pilot’ or ‘feasibility’ may be being incorrectly used to assist in the publication of small trials rather than to systematically test procedures to inform the conduct of a large definitive trial. Since those evaluations were published, the methods for evaluating the quality of reporting of pilot and feasibility trials have improved substantially with the introduction of an extension of the Consolidated Standards of Reporting Trials (CONSORT) statement specifically for randomised pilot and feasibility trials.^{8 9} This extension consists of a 40-item checklist for full articles and a 16-item checklist for abstracts.

The completeness of reporting of full published reports of randomised controlled trials^10–15 and the abstracts of trials^16–18 has been evaluated to be suboptimal across all areas of healthcare, including in physiotherapy.^{11 13–15 19} Factors that appear to be associated with improved reporting quality or methodological quality include publication in a journal with a high-impact factor,^20–22 being a multicentre trial,^{21 23–25} a higher number of authors,^{21 26} publication in a journal that endorses the CONSORT statement,^{12 13 19} language of the publication,^{15 21 25} discipline of physiotherapy,²⁷ year of publication,^{13 28} receiving grants from research funding agencies,¹⁴ sample size^{14 29} and evidence of clinical trial registration.^{13 19}

The number of randomised controlled trials in physiotherapy has grown exponentially over time.³⁰ Time and funding are resources that could be saved by conducting high-quality pilot and feasibility studies. To our knowledge, the reporting quality of pilot and feasibility trails of physiotherapy interventions using the new extension of the CONSORT statement for randomised pilot and feasibility trials has not yet been performed, and the factors associated with better reporting have not been identified.

Objectives

The purpose of this methodological survey is to describe the quality of reporting of abstracts and full articles of pilot or feasibility trials from a representative sample in the field of physiotherapy. Specifically, the first aim is to determine the percentage of trial reports indexed in the Physiotherapy Evidence Database (PEDro), which claim to be pilot or feasibility trials that evaluate feasibility. The second aim is to determine the aspect of feasibility evaluated in the primary objectives of the true pilot or feasibility trials. The third aim is to describe the completeness of reporting of abstracts and full articles using the CONSORT extension for randomised pilot and feasibility trials. The fourth aim is to investigate factors associated with completeness of reporting of pilot or feasibility trials.

Methods

Study design

This study is a methodological survey of completeness of reporting of abstracts and full articles of pilot or feasibility trials for physiotherapy interventions.

Eligibility criteria

We will include all reports of randomised controlled trials indexed in PEDro (http://www.pedro.org.au) that claim to be a pilot or feasibility trial. We will only include trials published in 2011–2017 that are fully indexed in PEDro (in-process trials, which have not had search terms and PEDro scores allocated, will not be included). We decided to only include trials published after 2010 because the International Committee of Medical Journal Editors stated that all trials started after July 2005 should be registered in a free, publicly available and electronically searchable register^{31 32} and also because the last update of the CONSORT statement was published in 2010. There will be no language restrictions.

We selected PEDro as the source of trial reports because PEDro is one of the most comprehensive indexes of reports of randomised controlled trials evaluating physiotherapy interventions.^{33 34} Moreover, all trials indexed on PEDro are rated for methodological quality and completeness of statistical reporting using the PEDro scale,³⁵ and are coded for the area (or subdiscipline) of physiotherapy practice and type of intervention. To be eligible for inclusion on PEDro, trials must involve comparison of at least two interventions (or an intervention and control condition) applied to subjects who are representative of those whom the interventions might be applied to in the course of clinical practice, with at least one of the interventions under evaluation being part of physiotherapy practice. In addition, all trials included in PEDro must involve random (or intended-to-be-random) allocations of subjects into interventions and must be fully published in a peer-reviewed journal.³⁶

Search strategy

To identify reports of pilot or feasibility trials, a search on PEDro will be conducted for the period from 2011 to 2017. We will use ’Clinical trial' in the Method field, combined with the following search terms in the Abstract and the Title field: Pilot* OR Feasibility* OR Vanguard* OR “Dress rehearsal”.

Study selection

Two independent reviewers will screen titles and abstracts to identify references that claim to be a pilot or feasibility trial. The title, abstract and, if necessary, full-text of these self-identified pilot or feasibility trials will be evaluated to identify the subset of articles that contain objectives linked to feasibility. Any disagreements between reviewers will be resolved by discussion or, if necessary, arbitration by a third reviewer. Figure 1 presents the flow diagram used to guide article selection.

Figure 1. Study flow diagram. PEDro, Physiotherapy Evidence Database.

Data extraction

Two independent reviewers will classify the reasons to conduct a pilot or feasibility for each included trial. There are several reasons for conducting pilot and feasibility studies. These reasons could be grouped under some broad classifications.³⁷ A widely known tutorial⁴ aimed to provide a detailed examination of the key aspects of pilot studies suggested four categories to classify the rationale to conduct a pilot study. The four categories are (1) process (steps that need to take place as part of the main study), (2) resources (time and budget), (3) management (human and data optimisation) and (4) scientific (issues such as treatment safety).⁴ If more than one category is identified for an included trial, we will code for all relevant categories and indicate which category is linked to the primary objective of the trial. The number of subjects randomised and whether the pilot or feasibility trial recommends that a large-scale trial will be conducted will also be recorded.

The two independent reviewers will also complete the CONSORT pilot and feasibility trials checklist (40 items) and the CONSORT pilot and feasibility trials abstracts checklist (16 items) (note that the ‘author’ item was omitted as this relates to conference abstracts only) for each trial.^{8 9} The CONSORT checklist includes items related to the title, trial design, methods, results, conclusions, registration and funding. Each item will be rated as ‘reported’, ‘inadequately reported’, ‘not reported’ or ‘not applicable’. Summary scores for the CONSORT pilot and feasibility trials checklist (range 0–40) and the CONSORT pilot and feasibility trials abstracts checklist (range 0–16) will be calculated by tallying the items scored as reported.

The reviewers will independently extract the data using an electronic data extraction form designed for this survey. The data extraction form will be created using information from the CONSORT extension to randomised pilot and feasibility trials. We will pilot the data extraction forms on 10 randomly selected trials before proceeding with full data extraction to ensure all reviewers extract data consistently and to ensure the data extraction form is unambiguous and free from errors. Discrepancies between the two reviewers will be resolved by discussion and by consulting the published explanation of the CONSORT checklists. If necessary, arbitration by a third reviewer will help to provide consensus on the data extracted. In order to improve the clarity regarding inclusions and exclusions and to increase accuracy and consistency among the reviewers, between-reviewer agreements will be measured using the kappa coefficients using an initial trial run involving 10 articles per reviewer. If adequate reliability will be not achieved, additional training or improvement in the data extraction form will be undertaken.

PEDro scale scores, subdiscipline of physiotherapy, intervention, language of publication and year of publication will be downloaded from PEDro. The PEDro scale is an 11-item scale that measures the methodological quality and completeness of statistical reporting of reports of randomised controlled trials.³⁵ The items are (1) eligibility criteria and source of subjects, (2) random allocation, (3) concealed allocation, (4) baseline comparability, (5) blinding of subjects, (6) blinding of therapists, (7) blinding of assessors, (8) >85% follow-up, (9) intention-to-treat analysis, (10) between-group statistical comparisons, and (11) reporting of point measures and measures of variability.³⁵ Each item is rated as ‘yes’ (unambiguously achieved) or ‘no’, with the number of yes responses for items 2–11 tallied to obtain the total PEDro score (out of 10). Both the individual items (coded as ‘0’ for no or ‘1’ for yes) and the total PEDro score (range 0–10) will be downloaded. There is evidence that the PEDro scale has higher reliability for individual ratings and consensus ratings compared with the Cochrane risk of bias.^{35 38} Also, the PEDro scale is strongly correlated (r=0.83, 95% CI 0.76 to 0.88) with the Cochrane risk-of-bias scale.³⁹ On the other hand, a meta-epidemiological study found discrepancies in terms of clinical trial quality using PEDro and the Cochrane risk-of-bias scale.⁴⁰ The subdiscipline of physiotherapy codes are cardiothoracic, continence and women’s health, ergonomics and occupational health, gerontology, musculoskeletal, neurology, oncology, orthopaedics, paediatrics and sports, or no appropriate value. Each trial can be assigned up to three codes for subdiscipline; in our study, we will select the most applicable subdiscipline and this will be coded as a dummy variable. The language of publication will be coded to produce two different variables: '1' for English and '0' for languages other than English, and Chinese as ‘0’ and all other languages as ‘1’. The year of publication will be subtracted from 2017 to produce an ‘age’ (in years) for each trial.

One reviewer will determine if the trial was registered and if the journal of publication for each trial endorses the CONSORT statement. Clinical trial registration will be extracted from the full article or, if not reported in the full article, by searching the International Clinical Trials Registry Platform (http://apps.who.int/trialsearch/) and will be coded as 1 for yes or 0 for no. Journal endorsement of the CONSORT statement will be achieved by reviewing the list of journals on the CONSORT website⁴¹ and, if necessary, by visiting journal websites and reviewing the instructions for authors and other editorial policies.

One reviewer will collect the journal impact factor if available at the time of pilot trial publication (as a continuous variable) through a search at the Journal Citation Reports website (https://jcr.incites.thomsonreuters.com). Other variables, including number of authors (as a continuous variable), source of funding, declaration of conflict of interests and sample size (as a continuous variable), will be collected by one reviewer through the electronic data extraction form designed for this review.

Patient and public involvement

Patients and the public were not involved in this study.

Statistical analysis

First, we will calculate the number, percentage and 95% CI of trials indexed in PEDro that claim to be a feasibility or pilot trial that evaluates feasibility. The PEDro CI calculator will be used to calculate the 95% CI.⁴² We will also compute the aspect of feasibility evaluated in the primary objectives of the pilot or feasibility trials.

The primary analysis will be a descriptive analysis of completeness of reporting of the abstracts and full articles of the pilot or feasibility trials. The frequency that each item is scored as reported, inadequately reported, not reported and not applicable for the CONSORT pilot and feasibility trials checklist and the CONSORT pilot and feasibility trials abstracts checklist will be tabulated. The mean (SD) summary score will be calculated for each checklist.

In the secondary analysis, we will perform a Poisson regression analysis to determine which study characteristics are associated with greater completeness reporting. Two independent models will be built, one using the summary score for the CONSORT pilot and feasibility trials checklist (ie, for the full article) as the dependent variable and the second model using the summary score for the CONSORT pilot and feasibility trials abstracts checklist. Independent variables for both models will be (1) publication in a journal that endorses CONSORT^{12 13} (1 for yes or 0 for no), (2) trial funded¹⁴ (1 for yes or 0 for no), (3) sample size¹⁴ (as a continuous variable), (4) reported trial registration number (1 for yes and 0 for no), (5) total PEDro score (continuous variable, 0–10), (6) most applicable subdiscipline of physiotherapy²⁷ (coded as dummy variables), (7) language of publication (1 for English and 0 for all other languages), (8) non-Chinese reports (1 for yes and 0 for ‘trials published in languages other than Chinese’), (9) number of authors (continuous variable) and (10) reporting allocation concealment (PEDro scale item 3, 1 for yes and 0 for no).

We will use generalised estimating equation (GEE) analysis, assuming an exchangeable correlation structure, to explore factors associated with completeness of reporting. GEE allows us to model a possible correlation or similarity of the papers published within the same journal. All analyses will be performed using SAS 9.2 (Cary, NC).

Discussion

This study will be the first to describe the completeness of reporting of pilot or feasibility trials from a representative sample in the field of physiotherapy using the CONSORT extension to randomised pilot and feasibility trials. This is important as good reporting, or transparency, will provide sufficient information about the methods and results of the trial to guide clinical practice and further research to both clinicians and researchers.

The transparency in reporting randomised controlled trials has improved since the introduction of the CONSORT statement.¹² A number of other factors are also associated with better trial quality, including being funded,¹⁴ being prospectively registered,¹³ being published in English¹⁵ and having larger sample sizes.^{14 29} Whether these variables are also associated with better reporting quality of pilot or feasibility studies has not been rigorously investigated. To the best of our knowledge, only one study has evaluated quality of reporting of pilot studies.⁷ That study did not use a scale or checklist to evaluate reporting nor did it test for possible factors that could predict quality.⁷

Footnote

Contributors LFCS wrote the first draft. LFCS, LdCMC and MA incorporated comments from authors on successive manuscripts. LT, AMM and LdCMC conceptualized the study. All authors contributed to the design of the study and reviewed and approved the manuscript.

Funding This study has not received external funding. LFCS has his Master by Research scholarship supported by the Coordination for the Improvement of Higher Education Personnel–Brazil (CAPES)–Finance Code 001. MA has his Postdoctoral Fellowship supported by the São Paulo Research Foundation, grant#2016/10317-0. The article processing charges was funded by the São Paulo Research Foundation, grant #2019/00812-1

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Patient consent for publication Not required.

References

¹ Loscalzo J. Pilot trials in clinical research: of what value are they? Circulation 2009; 119: 1694–6. doi:10.1161/CIRCULATIONAHA.109.861625

² Feeley N, Cossette S, Côté J, et al. The importance of piloting an RCT intervention. Can J Nurs Res 2009; 41: 85–99.

³ Eldridge S, Bond C, Campbell M, et al. Definition and reporting of pilot and feasibility studies. Trials 2013; 14: 1. doi:10.1186/1745-6215-14-S1-O18

⁴ Thabane L, Ma J, Chu R, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010; 10: 1. doi:10.1186/1471-2288-10-1

⁵ Eldridge SM, Lancaster GA, Campbell MJ, et al. Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework. PLoS One 2016; 11: e0150205. doi:10.1371/journal.pone.0150205

⁶ Kaur N, Figueiredo S, Bouchard V, et al. Where have all the pilot studies gone? A follow-up on 30 years of pilot studies in Clinical Rehabilitation. Clin Rehabil 2017; 31: 1238–48. doi:10.1177/0269215517692129

⁷ Kannan S, Gowri S. Pilot studies: Are they appropriately reported? Perspect Clin Res 2015; 6: 207–10. doi:10.4103/2229-3485.167097

⁸ Eldridge SM, Chan CL, Campbell MJ, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud 2016; 2: 64. doi:10.1186/s40814-016-0105-8

⁹ Thabane L, Hopewell S, Lancaster GA, et al. Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials. Pilot Feasibility Stud 2016; 2: 13. doi:10.1186/s40814-016-0065-z

¹⁰ Rikos D, Dardiotis E, Tsivgoulis G, et al. Reporting quality of randomized-controlled trials in multiple sclerosis from 2000 to 2015, based on CONSORT statement. Mult Scler Relat Disord 2016; 9: 135–9. doi:10.1016/j.msard.2016.07.013

¹¹ Riley SP, Swanson B, Brismée JM, et al. A systematic review of orthopaedic manual therapy randomized clinical trials quality. J Man Manip Ther 2016; 24: 241–52. doi:10.1080/10669817.2015.1119372

¹² Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 2012; 11: MR000030. doi:10.1002/14651858.MR000030.pub2

¹³ Geha NN, Moseley AM, Elkins MR, et al. The quality and reporting of randomized trials in cardiothoracic physical therapy could be substantially improved. Respir Care 2013; 58: 1899–906. doi:10.4187/respcare.02379

¹⁴ Maas E, Maher C, Moseley A, et al. Funding is related to the quality, conduct, and reporting of trial reports in musculoskeletal physical therapy: A survey of 210 published trials. Physiother Theory Pract 2016; 32: 628–35. doi:10.1080/09593985.2016.1222472

¹⁵ Shiwa SR, Moseley AM, Maher CG, et al. Language of publication has a small influence on the quality of reports of controlled trials of physiotherapy interventions. J Clin Epidemiol 2013; 66: 78–84. doi:10.1016/j.jclinepi.2012.08.004

¹⁶ Bigna JJ, Noubiap JJ, Asangbeh SL, et al. Abstracts reporting of HIV/AIDS randomized controlled trials in general medicine and infectious diseases journals: completeness to date and improvement in the quality since CONSORT extension for abstracts. BMC Med Res Methodol 2016; 16: 138. doi:10.1186/s12874-016-0243-y

¹⁷ Cui Q, Tian J, Song X, et al. Does the CONSORT checklist for abstracts improve the quality of reports of randomized controlled trials on clinical pathways? J Eval Clin Pract 2014; 20: 827–33. doi:10.1111/jep.12200

¹⁸ Kuriyama A, Takahashi N, Nakayama T. Reporting of critical care trial abstracts: a comparison before and after the announcement of CONSORT guideline for abstracts. Trials 2017; 18: 32. doi:10.1186/s13063-017-1786-x

¹⁹ Gonzalez GZ, Moseley AM, Maher CG, et al. Methodologic Quality and Statistical Reporting of Physical Therapy Randomized Controlled Trials Relevant to Musculoskeletal Conditions. Arch Phys Med Rehabil 2018; 99: 129–36. doi:10.1016/j.apmr.2017.08.485

²⁰ Montané E, Vallano A, Vidal X, et al. Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: a systematic review. BMC Clin Pharmacol 2010; 10: 2. doi:10.1186/1472-6904-10-2

²¹ Samaan Z, Mbuagbaw L, Kosa D, et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc 2013; 6: 169–88. doi:10.2147/JMDH.S43952

²² Ghimire S, Kyung E, Lee H, et al. Oncology trial abstracts showed suboptimal improvement in reporting: a comparative before-and-after evaluation using CONSORT for Abstract guidelines. J Clin Epidemiol 2014; 67: 658–66. doi:10.1016/j.jclinepi.2013.10.012

²³ Sriganesh K, Bharadwaj S, Wang M, et al. Quality of abstracts of randomized control trials in five top pain journals: a systematic survey. Contemp Clin Trials Commun 2017; 7: 64–8. doi:10.1016/j.conctc.2017.06.001

²⁴ Mbuagbaw L, Thabane M, Vanniyasingam T, et al. Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review. Contemp Clin Trials 2014; 38: 245–50. doi:10.1016/j.cct.2014.05.012

²⁵ Zhong Y, Zhou W, Jiang H, et al. Quality of reporting of two-group parallel randomized controlled clinical trials of multi-herb formulae: A survey of reports indexed in the Science Citation Index Expanded. Eur J Integr Med 2011; 3: e309–e316. doi:10.1016/j.eujim.2011.09.005

²⁶ Balasubramanian SP, Wiener M, Alshameeri Z, et al. Standards of reporting of randomized controlled trials in general surgery: can we do better? Ann Surg 2006; 244: 663–7. doi:10.1097/01.sla.0000217640.11224.05

²⁷ Moseley AM, Elkins MR, Janer-Duncan L, et al. The Quality of Reports of Randomized Controlled Trials Varies between Subdisciplines of Physiotherapy. Physiother Can 2014; 66: 36–43. doi:10.3138/ptc.2012-68

²⁸ Moseley AM, Herbert RD, Maher CG, et al. Reported quality of randomized controlled trials of physiotherapy interventions has improved over time. J Clin Epidemiol 2011; 64: 594–601. doi:10.1016/j.jclinepi.2010.08.009

²⁹ Rios LP, Odueyungbo A, Moitri MO, et al. Quality of reporting of randomized controlled trials in general endocrinology literature. J Clin Endocrinol Metab 2008; 93: 3810–6. doi:10.1210/jc.2008-0817

³⁰ Kamper SJ, Moseley AM, Herbert RD, et al. 15 years of tracking physiotherapy evidence on PEDro, where are we now? Br J Sports Med 2015; 49: 907–9. doi:10.1136/bjsports-2014-094468

³¹ De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med 2004; 351: 1250–1. doi:10.1056/NEJMe048225

³² Costa LO, Lin CW, Grossi DB, et al. Clinical trial registration in physical therapy journals: Recommendations from the International Society of Physiotherapy Journal Editors. Phys Ther 2013; 93: 6–10. doi:10.2522/ptj.2013.93.1.6

³³ Michaleff ZA, Costa LO, Moseley AM, et al. CENTRAL, PEDro, PubMed, and EMBASE are the most comprehensive databases indexing randomized controlled trials of physical therapy interventions. Phys Ther 2011; 91: 190–7. doi:10.2522/ptj.20100116

³⁴ Moseley AM, Sherrington C, Elkins MR, et al. Indexing of randomised controlled trials of physiotherapy interventions: a comparison of AMED, CENTRAL, CINAHL, EMBASE, hooked on evidence, PEDro, PsycINFO and PubMed. Physiotherapy 2009; 95: 151–6. doi:10.1016/j.physio.2009.01.006

³⁵ Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther 2003; 83: 713–21.

³⁶ Costa LO, Moseley AM, Sherrington C, et al. Core journals that publish clinical trials of physical therapy interventions. Phys Ther 2010; 90: 1631–40. doi:10.2522/ptj.20090419

³⁷ Van Teijlingen ER, Rennie AM, Hundley V, et al. The importance of conducting and reporting pilot studies: the example of the Scottish Births Survey. J Adv Nurs 2001; 34: 289–95. doi:10.1046/j.1365-2648.2001.01757.x

³⁸ Shiwa SR, Costa LO, Costa LC, et al. Reproducibility of the Portuguese version of the PEDro Scale. Cad Saude Publica 2011; 27: 2063–8. doi:10.1590/S0102-311X2011001000019

³⁹ Yamato TP, Maher C, Koes B, et al. The PEDro scale had acceptably high convergent validity, construct validity, and interrater reliability in evaluating methodological quality of pharmaceutical trials. J Clin Epidemiol 2017; 86: 176–81. doi:10.1016/j.jclinepi.2017.03.002

⁴⁰ Armijo-Olivo S, da Costa BR, Cummings GG, et al. PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study. PLoS One 2015; 10: e0132634. doi:10.1371/journal.pone.0132634

⁴¹ CONSORT. Endorsers journals and organizations. 2017. http://www.consort-statement.org/about-consort/endorsers

⁴² PEDro. Physiotherapy Evidence Database Australia. 2017. https://www.pedro.org.au/english/downloads/confidence-interval-calculator/