How Psychiatry Journals Support the Unbiased Translation of Clinical Research. A Cross-Sectional Study of Editorial Policies

Abstract

Introduction

Reporting guidelines (e.g. CONSORT) have been developed as tools to improve quality and reduce bias in reporting research findings. Trial registration has been recommended for countering selective publication. The International Committee of Medical Journal Editors (ICMJE) encourages the implementation of reporting guidelines and trial registration as uniform requirements (URM). For the last two decades, however, biased reporting and insufficient registration of clinical trials has been identified in several literature reviews and other investigations. No study has so far investigated the extent to which author instructions in psychiatry journals encourage following reporting guidelines and trial registration.

Method

Psychiatry Journals were identified from the 2011 Journal Citation Report. Information given in the author instructions and during the submission procedure of all journals was assessed on whether major reporting guidelines, trial registration and the ICMJE’s URM in general were mentioned and adherence recommended.

Results

We included 123 psychiatry journals (English and German language) in our analysis. A minority recommend or require 1) following the URM (21%), 2) adherence to reporting guidelines such as CONSORT, PRISMA, STROBE (23%, 7%, 4%), or 3) registration of clinical trials (34%). The subsample of the top-10 psychiatry journals (ranked by impact factor) provided much better but still improvable rates. For example, 70% of the top-10 psychiatry journals do not ask for the specific trial registration number.

Discussion

Under the assumption that better reported and better registered clinical research that does not lack substantial information will improve the understanding, credibility, and unbiased translation of clinical research findings, several stakeholders including readers (physicians, patients), authors, reviewers, and editors might benefit from improved author instructions in psychiatry journals. A first step of improvement would consist in requiring adherence to the broadly accepted reporting guidelines and to trial registration.

About the Authors:

Hannes Knüppel

Affiliation: Institute of History and Ethics in Medicine, Centre for Ethics and Law in the Life Sciences – CELLS, Hannover Medical School, Germany

Courtney Metz

Affiliations Institute of History and Ethics in Medicine, Centre for Ethics and Law in the Life Sciences – CELLS, Hannover Medical School, Germany, Department of Philosophy, Centre for Ethics and Law in the Life Sciences – CELLS, Leibniz University of Hannover, Germany

Joerg J. Meerpohl

Affiliation: German Cochrane Centre, Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Germany

Daniel Strech

* E-mail: [email protected]

Affiliation: Institute of History and Ethics in Medicine, Centre for Ethics and Law in the Life Sciences – CELLS, Hannover Medical School, Germany

Abstract

Introduction

Reporting guidelines (e.g. CONSORT) have been developed as tools to improve quality and reduce bias in reporting research findings. Trial registration has been recommended for countering selective publication. The International Committee of Medical Journal Editors (ICMJE) encourages the implementation of reporting guidelines and trial registration as uniform requirements (URM). For the last two decades, however, biased reporting and insufficient registration of clinical trials has been identified in several literature reviews and other investigations. No study has so far investigated the extent to which author instructions in psychiatry journals encourage following reporting guidelines and trial registration.

Method

Psychiatry Journals were identified from the 2011 Journal Citation Report. Information given in the author instructions and during the submission procedure of all journals was assessed on whether major reporting guidelines, trial registration and the ICMJE’s URM in general were mentioned and adherence recommended.

Results

We included 123 psychiatry journals (English and German language) in our analysis. A minority recommend or require 1) following the URM (21%), 2) adherence to reporting guidelines such as CONSORT, PRISMA, STROBE (23%, 7%, 4%), or 3) registration of clinical trials (34%). The subsample of the top-10 psychiatry journals (ranked by impact factor) provided much better but still improvable rates. For example, 70% of the top-10 psychiatry journals do not ask for the specific trial registration number.

Discussion

Under the assumption that better reported and better registered clinical research that does not lack substantial information will improve the understanding, credibility, and unbiased translation of clinical research findings, several stakeholders including readers (physicians, patients), authors, reviewers, and editors might benefit from improved author instructions in psychiatry journals. A first step of improvement would consist in requiring adherence to the broadly accepted reporting guidelines and to trial registration.

Figures

Table 4

Table 1

Table 2

Table 3

Table 4

Table 1

Table 2

Table 3

Citation: Knüppel H, Metz C, Meerpohl JJ, Strech D (2013) How Psychiatry Journals Support the Unbiased Translation of Clinical Research. A Cross-Sectional Study of Editorial Policies. PLoS ONE 8(10): e75995. https://doi.org/10.1371/journal.pone.0075995

Editor: Neil R. Smalheiser, University of Illinois-Chicago, United States of America

Received: June 26, 2013; Accepted: August 16, 2013; Published: October 16, 2013

Copyright: © 2013 Knüppel et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: The authors have no support or funding to report.

Competing interests: The authors have declared that no competing interests exist.

Background

The successful translation of findings from clinical trials into health care practice, guidelines and patient information depends on the timely, accurate and unbiased reporting of trial methodology and results. The quality and reporting of clinical trials and systematic reviews can, however, be sub-optimal. Even within the design of RCTs, for example, there is the inherent risk of bias skewing results at various stages and minimizing internal and external validity [1].

First, there is empirical evidence to suggest that lack of, or inadequate attention to, random allocation, allocation concealment, blinding and intention to treat can lead to bias [2], [3]. Second, setting, participants, demographic data, co-medication e.g. can limit the generalizability of the trial results [4], [5]. There is also increasing evidence of selective reporting in clinical trial findings, with some recent examples in pharmacologic treatment for depression and other psychiatric disorders [6], [7], [8], [9].

Since the early 1990s, medical journal editors, methodologists, and clinical researchers have developed reporting guidelines as tools to help improve the quality of reporting in health research articles. A reporting guideline is a checklist, flow diagram, or explicit text to guide authors in reporting a specific type of research, developed using explicit methodology [10]. The first guideline, the CONSORT (CONsolidated Standards Of Reporting Trials) statement, was developed to improve quality of reports on randomized controlled trials; it was first published in 1996, revised in 2001, and updated in 2010 [11], [12]. Reporting guidelines are also available for various other study designs, including diagnostic test accuracy studies (STAndards for Reporting Diagnostic accuracy, STARD) [13], observational studies (STrengthening the Reporting of Observational studies in Epidemiology, STROBE) [14], Meta-analysis Of Observational Studies in Epidemiology (MOOSE) [15], and systematic reviews of randomized controlled trials (Preferred Reporting Items for Systematic reviews and Meta-Analyses, PRISMA) [16].

A recent review of 134 RCTs on pharmacological treatment of bipolar disorder published between 2000 and 2010 found that while some trial-related information is well reported a good part of the reporting quality of RCTs in bipolar disorder falls well below the required level as aimed for by CONSORT [17], [18]. Twenty-five percent (n = 18) of all CONSORT items were generally reported inadequately (reported adequately in less than 25% of all trials). These neglected parts include essential methodological items such as the generation of random allocation sequence (reported in only 24% of all RCTs), method of allocation concealment (in 22%), and all items relevant to the randomization implementation. Also, information with essential clinical relevance was generally reported inadequately, such as the effect size (in 22%) and the number needed to treat (16%). Other analyses of the quality of reporting in psychiatry journals have made similar findings [19], [20], [21].

The poor quality of reporting combined with the selective reporting of trial findings undermines timely, accurate and unbiased translation of trial results in health care practice. It has been shown, firstly, that entire trials with primarily negative results were not published at all (publication bias) [22]. Secondly, it has more recently been shown that some published trials report information selectively, with the effect of prioritizing the benefit of a medical measure or suppressing the results concerning its potential harm[23], [24]. There is a consensus in medical research, in publication ethics and among the leading scientific journals that trial registration currently represents the best strategy for countering selective publication or making it suitably transparent [24], [25], [26]. Trial registers have existed since the 1960s [27]. The most significant registries at present are ClinicalTrials.gov, run by the National Library of Medicine (USA), which has been accepting clinical trials outside the USA since 2005, and the registry network of the WHO, the International Clinical Trials Registry Platform (ICTRP), which has been in operation since 2007.

The Uniform Requirements for Manuscripts Submitted to Biomedical Journals (URM) developed by the International Committee of Medical Journal Editors (ICJME) [28] require, first, that authors consult reporting guidelines relevant to their specific research design (such as CONSORT for RCTs, or other tools that can be identified at the website of the EQUATOR network (www.equator-network.org) and, second, that trials are registered in a public trials registry.

While the responsibility for improvement of unbiased reporting should primarily lie with the investigators, reviewers and journal editors could facilitate the process by encouraging authors to consider reporting guidelines and to register their trials. Whether reporting guidelines are being endorsed and implemented by medical journals has been studied for general medicine [29], [30], pediatrics and urology [31], [32], [33], [34].

Although inadequate quality of reporting and selective reporting of trial data have often and recently been demonstrated for psychiatric disorders [6], [7], [8], [9], [17], [19], [20], [21], no study has so far investigated the extent to which author instructions in psychiatry journals endorse reporting guidelines and trial registration as encouraged by the URM.

This study aimed to analyze whether author instructions and instructions during the submission procedure of psychiatry journals mention, recommend, or require 1) the adherence to the URM as published by the ICMJE; 2) the use of major reporting guidelines; and 3) trial registration.

Methods

Based on Journal Citation Reports from 2011 we identified 130 journals indexed in the subject category “psychiatry”. We also identified a subsample of 10 psychiatry journals with the highest impact factor (“top-10”). We restricted our analysis to Journals published in English or German. We accessed the “author’s instructions” or similar texts on the journals’ websites between July and August 2012. We further accessed the instructions given during the online submission procedure in September 2012. The online submission procedures were entered by a fake submission of an “original paper” or a “clinical research”, “clinical trial” paper. All PDFs or website texts were downloaded using WinHTTrack 3.46-1 for documentation.

Two authors independently assessed whether the author instructions mention the URM, major reporting guidelines (CONSORT, STARD, STROBE, MOOSE, and PRISMA) and trial registration. The QUOROM (QUality Of Reporting Of Meta-analyses) guideline was updated and renamed PRISMA in 2009; for this analysis, we classified QUOROM as a subgroup of PRISMA. The rating options were 1) “not mentioned”, 2) “mentioned” (without specification) 3) “consideration recommended” or 4) “consideration required”. The rating “consideration recommended” was applied to moderate wording in the author instructions such as “should” or “we recommend that…”. The rating “consideration required” was applied to strong wording like “authors must …”, “we expect authors to …” or “we require authors to…”.

If two or more journals referred to the same author instructions (e.g. because of the same publisher), they were treated as independent journals for evaluation.

We accessed the ICMJE website in September 2012 to identify which journals are listed as following the URMs.

We calculated frequency data by standard descriptive statistics.

Results

After exclusion of 7 psychiatry journals due to language restriction or the lack of any web page we included 123 journals in our analysis (116 in English and 7 in German language).

Author’s instructions regarding URM and ICMJE policies

From the 123 psychiatry journals 21% (n = 26) recommend or require following the URM and another 34% (n = 42) “only” mention the URM at some point in their author instructions or during the online submission process (see table 1).

[Figure omitted. See PDF.]

Table 1. Author instructions regarding the Uniform Requirements for Manuscripts (URM) developed by the ICMJE.

https://doi.org/10.1371/journal.pone.0075995.t001

In contrast, 90% of the top-10 psychiatry journals recommend or require adherence to ICMJE’s URMs.

Of the 123 psychiatry journals, 11 are listed on the ICMJE website among other journals that have requested inclusion on the list of publications that follow the ICMJE’s URMs. However, 2 of these 11 journals mention neither reporting guidelines nor trial registration in their author instructions or during their online submission process.

Author’s instructions regarding reporting guidelines

The CONSORT statement, which guides the reporting of randomized controlled trials (RCTs), was most prominently mentioned in the journals’ author instructions (see table 2).

[Figure omitted. See PDF.]

Table 2. Authors instructions regarding reporting guidelines.

https://doi.org/10.1371/journal.pone.0075995.t002

For all psychiatry journals 23% (n = 28) and for the top-10 psychiatry journals 50% either recommended or required adherence to CONSORT. All other reporting guidelines were recommended or required in 3% to 7% of all psychiatry journals and in 0% to 20% of the top-10 psychiatry journals (see table 2).

Author’s instructions regarding trial registration

Of all 123 psychiatry journals, 34% (n = 42) and for the top-10 psychiatry journals 70% explicitly recommend or require the authors to register clinical trials. Only 13 of these (11% of all 123 journals and 30% of the top-10 journals) require the registration number during their online submission process (see table 3).

[Figure omitted. See PDF.]

Table 3. Author instructions regarding trial registration.

https://doi.org/10.1371/journal.pone.0075995.t003

Furthermore, only 12% (n = 15) of all psychiatry journals and 60% of the top-10 journals mention specific trial registries. In total, eleven different trial registries were mentioned with clinicaltrials.gov as the most prominent (n = 14).

Comparison among clinical specialities

The results for all psychiatry journals are similar to overview findings in other specialities like paediatrics and urology that applied assessment tools similar to those applied in this study [31], [32], [33], [34]. One author of this study (JM) also contributed to the editorial policy analyses in paediatrics and urology (see table 4).

[Figure omitted. See PDF.]

Table 4. Comparison of findings among clinical specialties.

https://doi.org/10.1371/journal.pone.0075995.t004

Discussion

Several internationally agreed policies and tools aim to improve the unbiased translation of research findings into clinical practice and health policy decision-making. Core policies and tools in this respect are 1) the URMs (uniform requirements for manuscripts submitted to biomedical journals) drafted by the ICMJE, 2) reporting guidelines (e.g. CONSORT, STROBE, PRISMA) collated by the EQUATOR network, and 3) trial registries such as clinicaltrials.gov run by the United States National Library of Medicine (NLM) at the National Institutes of Health or registries certified by the WHO and working with the International Clinical Trials Registry Platform (ICTRP).

Our main finding is that only a minority of all psychiatry journals (n = 123) recommend or require 1) following the URM (21%), 2) adherence to reporting guidelines such as CONSORT, PRISMA, STROBE (23%, 7%, 4%), or 3) registration of clinical trials (34%). While the top-10 psychiatry journals (ranked by impact factor) highlight core recommendations and requirements more frequently (URM = 90%, CONSORT = 50%, trial registration = 70%) there is still room for improvement. Beside the fact that three top-10 journals do not recommend or require trial registration only one top-10 journal recommends or requires authors to follow the PRISMA statement that aims to support reporting of systematic reviews of clinical trials.

It is obvious that authors are accountable for their manuscripts, and it is their obligation to prepare their research articles in an accurate, transparent, and complete manner so that all the information important for data interpretation is available. However, we suspect that many authors will neither know the recommendations given in the ICMJE’s URMs, nor reporting guidelines such as CONSORT or the practical and ethical rationale for registering clinical trials.

One first reason for scientific journals to include information in their author instructions about reporting guidelines and trial registries is to help potential authors to refine the scientific strength and impact of their publications. Authors are not only interested in the publication of papers. Academic remits more and more refer to how the scientific community judges the content of papers. For example, post-publication reviews and the number of citations are becoming more important for academic careers and grant proposals. Nevertheless, beside the intrinsic motivation of researchers the unbiased translation of research findings also depends on its consistent and rigor promotion. Thus, strong wording in editorial policies that require trial registration and the application of reporting guidelines is necessary but not sufficient. The adherence to such requirements should be made verifiable, for example by requiring the inclusion of the trial registration number in the manuscript. At present, however, 70% of the high impact journals in Psychiatry do not ask for the specific trial registration number.

Furthermore, it is questionable whether the peer-review process is sufficient to guarantee completeness and accuracy of funded research [35] and good reporting quality [31]. Because better structured papers that do not lack substantial information can improve readability, reviewers and readers might also benefit from author instructions that help to improve reporting quality.

As well as authors, journals might also have an interest in adhering to internationally agreed and broadly accepted quality standards. We currently face controversial discussions about the best way to organize scientific publication. Public institutions discuss whether to sponsor open access publications. Against this background, journals that do not support and promote basic measures to improve the readability and credibility of publications may struggle to remain viable in the near future. Public financing of open access publications should require that journals which classify for reimbursement of publication fees include information about reporting guidelines and trial registration in their author instructions and during their online submission process.

Independently of the personal interests of researchers and journal editors, good science should primarily aim to decrease biased publications of information that can negatively influence clinical and public health decision-making. For example, the validity of systematic reviews and meta-analyses that synthesize findings from original studies will be undermined by biased or poorly reported research findings. Finally, the core principles of medicine (including the Ethics Codex of the APA) such as non-maleficence, respect of autonomy and justice all demand greater efforts by journal editors to improve the quality of reporting and trial registration [36].

For the field of psychiatry, which addresses an immense patient population with one of the world’s highest burdens of disease, major improvements have to be made with respect to how the majority of journals inform and require their authors to adhere to a high quality of reporting and adequate trial registration. Our review indicates that the top-10, high impact psychiatry journals demonstrate more interest in high quality publications. But also among these flagships of psychiatry journals more could be done to enforce the registration, improve the reporting, and finally facilitate unbiased translation of clinical research findings.

Author Contributions

Conceived and designed the experiments: HK JJM DS. Performed the experiments: HK CM DS. Analyzed the data: HK DS. Contributed reagents/materials/analysis tools: HK JJM DS. Wrote the paper: HK JJM DS.

Citation: Knüppel H, Metz C, Meerpohl JJ, Strech D (2013) How Psychiatry Journals Support the Unbiased Translation of Clinical Research. A Cross-Sectional Study of Editorial Policies. PLoS ONE 8(10): e75995. https://doi.org/10.1371/journal.pone.0075995

References

1. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, et al. (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. The CONSORT Group. Ann Intern Med 134: 663–694.

2. Juni P, Altman DG, Egger M (2001) Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 323: 42–46.

3. Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408–412.

4. Dans AL, Dans LF, Guyatt GH, Richardson S (1998) Users' guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. Evidence-Based Medicine Working Group. JAMA 279: 545–549.

5. Laupacis A, Sackett DL, Roberts RS (1988) An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 318: 1728–1733.

6. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003) Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326: 1171–1173.

7. McHenry L (2006) Ethical issues in psychopharmacology. J Med Ethics 32: 405–410.

8. Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, et al. (2004) Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet 363: 1341–1345.

9. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.

10. Moher D, Weeks L, Ocampo M, Seely D, Sampson M, et al. (2011) Describing reporting guidelines for health research: a systematic review. J Clin Epidemiol 64: 718–742.

11. Moher D, Jones A, Lepage L (2001) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA 285: 1992–1995.

12. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, et al. (2010) CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 340: c869.

13. Brody H (2012) From an ethics of rationing to an ethics of waste avoidance. The New England Journal of Medicine 366: 1949–1951.

14. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 147: 573–577.

15. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, et al. (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 283: 2008–2012.

16. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097.

17. Strech D, Soltmann B, Weikert B, Bauer M, Pfennig A (2011) Quality of reporting of randomized controlled trials of pharmacologic treatment of bipolar disorders: a systematic review. The Journal of clinical psychiatry 72: 1214–1221.

18. Soltmann B, Pfennig A, Weikert B, Bauer M, Strech D (2012) [Quality of reporting in studies on bipolar disorders: implications for the development of guidelines]. Nervenarzt 83: 604–617.

19. Thornley B, Adams C (1998) Content and quality of 2000 controlled trials in schizophrenia over 50 years. BMJ 317: 1181–1184.

20. Cipriani A, Malvini L, Furukawa TA, Barbui C (2007) Relationship between quality of reports of antidepressant randomized controlled trials and treatment estimates: systematic review, meta-analysis, and meta-regression analysis. J Clin Psychopharmacol 27: 352–356.

21. Han C, Kwak KP, Marks DM, Pae CU, Wu LT, et al. (2009) The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry. Contemp Clin Trials 30: 116–122.

22. Dickersin K (1997) How important is publication bias? A synthesis of available data. AIDS Educ Prev 9: 15–21.

23. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291: 2457–2465.

24. Rising K, Bacchetti P, Bero L (2008) Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med 5: e217; discussion e217.

25. Levin LA, Palmer JG (2007) Institutional review boards should require clinical trial registration. Arch Intern Med 167: 1576–1580.

26. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, et al. (2004) Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Ann Intern Med 141: 477–478.

27. Dickersin K, Rennie D (2003) Registering clinical trials. JAMA 290: 516–523.

28. ICMJE (2008) Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication. International Committee of Medical Journal Editors.

29. Altman DG (2005) Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors. BMJ 330: 1056–1057.

30. Hopewell S, Altman DG, Moher D, Schulz KF (2008) Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of journal editors and journal 'Instructions to Authors'. Trials 9: 20.

31. Kunath F, Grobe HR, Rucker G, Engehausen D, Antes G, et al. (2012) Do journals publishing in the field of urology endorse reporting guidelines? A survey of author instructions. Urologia internationalis 88: 54–59.

32. Meerpohl JJ, Wolff RF, Niemeyer CM, Antes G, von Elm E (2010) Editorial policies of pediatric journals: survey of instructions for authors. Archives of pediatrics & adolescent medicine 164: 268–272.

33. Meerpohl JJ, Wolff RF, Antes G, von Elm E (2011) Are pediatric Open Access journals promoting good publication practice? An analysis of author instructions. BMC Pediatr 11: 27.

34. Kunath F, Grobe HR, Keck B, Rucker G, Wullich B, et al. (2011) Do urology journals enforce trial registration? A cross-sectional study of published trials. BMJ open 1: e000430.

35. Demicheli V, Di Pietrantonj C (2007) Peer review for improving the quality of grant applications. Cochrane Database Syst Rev: MR000003.

36. Strech D (2012) Normative arguments and new solutions for the unbiased registration and publication of clinical trials. Journal of Clinical Epidemiology 65: 276–281.