Press Releases Issued by Supplements Industry Organisations and Non-Industry Organisations in Response to Publication of Clinical Research Findings: A Case-Control Study

Abstract

Background

Dietary supplement use is increasing despite lack of evidence of benefits, or evidence of harm. Press releases issued by the supplements industry might contribute to this situation by using ‘spin’ (strategies to hype or denigrate findings) to distort the results of clinical studies. We assessed press releases issued in response to publication of clinical studies on dietary supplements.

Methods and Findings

We analyzed 47 supplements industry press releases and 91 non-industry press releases and news stories, generated in response to 46 clinical studies of dietary supplements published between 1/1/2005 and 5/31/2013. The primary outcome was ‘spin’ content and direction. We also assessed disposition towards use of dietary supplements, reporting of study information, and dissemination of industry press releases. More supplements industry press releases (100%) contained ‘spin’ than non-industry media documents (55%, P<0.001). Hyping ‘spin’ scores were higher in industry than non-industry media documents for studies reporting benefit of supplements (median ‘spin’ score 3.3, 95% CI 1.0–5.5 vs 0.5, 0–1.0; P<0.001). Denigratory ‘spin’ scores were higher in industry than non-industry media documents for studies reporting no effect (6.0, 5.0–7.0 vs 0, 0–0; P<0.001) or harm (6.0, 5.5–7.5 vs 0, 0–0.5; P<0.001) from a supplement. Industry press releases advocated supplement use in response to >90% of studies that reported no benefit, or harm, of the supplement. Industry press releases less frequently reported study outcomes, sample size, and estimates of effect size than non-industry media documents (all P<0.001), particularly for studies that reported no benefit of supplements. Industry press releases were referenced by 148 news stories on the websites of 6 organizations that inform manufacturers, retailers and consumers of supplements.

Conclusions

Dietary supplements industry press releases issued in response to clinical research findings are characterized by ‘spin’ that hypes results that are favourable to supplement use and denigrates results that are not.

About the Authors:

Michael T. M. Wang

Affiliation: Department of Medicine, University of Auckland, Auckland, New Zealand

Greg Gamble

Affiliation: Department of Medicine, University of Auckland, Auckland, New Zealand

Mark J. Bolland

Affiliation: Department of Medicine, University of Auckland, Auckland, New Zealand

Andrew Grey

* E-mail: [email protected]

Affiliation: Department of Medicine, University of Auckland, Auckland, New Zealand

Introduction

About half of US adults, and two-thirds of those>60 years, take dietary supplements [1]. Similar data have been reported outside of the USA [2], [3]. The supplements industry is profitable: Americans spend more than US$30 billion annually on dietary supplements [1]. Motivations to take dietary supplements are diverse, but users most commonly cite a wish to improve or maintain health [1], [4]. The sources of information which influence decisions to use dietary supplements are also multiple. Only 23% of US adults who take supplements do so on the advice of a health care professional [1]. In healthy older adults, important sources of information that influence decisions about supplement use include magazines, news articles, and people other than health professionals [2], [5].

In the past decade, there has been intensive investigation of the health benefits and risks of dietary supplements. Consequently, there have been many publications in prominent medical journals on dietary supplements; these often reported no benefit and sometimes reported harm [1], [6]–[8]. Publications of randomized clinical trial data showing no health benefit of omega-3 fatty acids had no discernible effect on the contemporaneous progressive increase in use of the supplements [9]. The reason(s) for the burgeoning use of dietary supplements despite accrual of rigorous evidence of no benefit or harm is uncertain, but a survey of supplement users reported that only 25% of users would alter their behaviour in response to findings of clinical studies that contradicted the health claims made by supplements manufacturers [4].

Organizations that represent the commercial interests of supplements manufacturers take an interest in the outcomes of clinical research on dietary supplements. A means by which organizations with commercial interests might influence the responses of supplements users to the outcomes of clinical research is via press releases that generate news stories in media accessed by marketers and consumers of supplements. To investigate this possibility, we analyzed the tone, content, conclusions, and propagation of press releases generated by prominent organizations representing the dietary supplements industry in response to the publication of clinical research about supplements. We also compared the industry press releases to contemporaneous non-industry press releases or news stories.

Materials and Methods

Study Documents

Between 5/31/13 and 6/15/13, we extracted from 3 industry websites (the Council for Responsible Nutrition, CRN, “the leading trade association representing dietary supplement manufacturers and ingredient suppliers”, http://www.crnusa.org, the Alliance for Natural Health,ANH, “a non-governmental organisation promoting natural and sustainable approaches to healthcare worldwide”, http://www.anh-usa.org, and the Natural Products Association,NPA, “the leading representative of the dietary supplement industry”, http://www.npainfo.org) press releases issued in response to clinical studies of dietary supplements published between 1/1/05 and 5/31/13. We defined a clinical study as one that assessed the effects on human health of a dietary supplement or its food-based equivalent.

For each publication that generated a press release from an industry source, we collated two press releases or news stories from non-industry sources, using a structured approach. First, we extracted press releases from the National Institutes Health (NIH)/National Center for Complementary and Alternative Medicine (NCCAM), by searching its website and the EurekAlert! database (http://www.eurekalert.org). Next, we collated press releases issued by the journals that published the source articles, by searching the journal websites and the EurekAlert! database. Lastly, we accessed news stories from news agencies, identified by a Google search using search terms that included the name of the supplement, the primary author of the source article, and the journal of publication, with a time limit of two months from the date of publication. We extracted the first 1–2 news stories that were identified by our search. If a press release referred to more than one source publication, it was analysed for each publication. If fewer than two corresponding non-industry media documents were identified, the industry press release was excluded from further analysis.

The publications that stimulated the press releases from industry sources were collated using PubMed.

Figure 1 summarizes the collation of study documents.

[Figure omitted. See PDF.]

Figure 1. Collation of study documents.

CRN, Council for Responsible Nutrition; NPA, Natural Products Association; ANH, Alliance for Natural Health.

https://doi.org/10.1371/journal.pone.0101533.g001

Data Abstraction

Data were abstracted independently by two reviewers (MW and AG). For data that involved some subjectivity, such as the presence, type, and direction of ‘spin’, the interpretation of the title and text of the media documents and the ‘spin’ score, differences were resolved by consensus.

Source Publications.

We adapted a data abstraction tool that evaluated the quality of reporting of clinical research [10], [11] (Text S1). Study conclusions were categorized as favours supplement use; does not favour supplement use - no effect; or does not favour supplement use - harm. If the source publication reported more than one outcome and the effects of the intervention were mixed, the assignation of study conclusion was to the category of the predominant effect or, if either benefit or harm was reported in combination with no effect, to favours supplement use or does not favour supplement use - harm, respectively.

Press Releases.

From the press releases and news stories, data were extracted on reporting of study characteristics (Text S2). Quotes from industry staff, independent experts, and study investigators were collated. The title and text of each media document were assessed as to whether they supported use of the supplement, did not support use of the supplement, or were neutral [10]. If one reviewer considered a document to be neutral towards supplement use and the other reviewer considered it to be either supportive or not supportive of supplement use, the document was scored as supporting or not supporting use, respectively.

‘Spin’ assessment.

Each press release or news story was assessed for the presence and amount of ‘spin’ considered to be either hyping of, or denigratory towards, the source publication, using a standardized format (Text S3). Hyping ‘spin’ is specific reporting that unduly emphasizes or exaggerates the benefit of an experimental treatment [12]. We defined denigratory ‘spin’ as specific reporting that unduly downplays or dismisses the lack of benefit, or harm, caused by an experimental treatment. To assess hyping ‘spin’ we adapted published methods which included a checklist of ‘spin’ strategies [10], [13]. To assess denigratory ‘spin’ we designed a data abstraction form by adapting that used for assessment of hyping ‘spin’ and incorporating into the checklist commonly employed denigratory strategies identified from a review of relevant literature [14], [15] and media and academic responses to publication of clinical research.

Each press release and news story in the study sample set was independently assessed by two reviewers (MW and AG). If a document was considered to either hype or denigrate the clinical research, the assessor proceeded to complete a checklist of ‘spin’ techniques (Text S3). To produce a score for each form of ‘spin’, we assigned one point to each checklist item identified in a press release or news story. Summing the hyping and denigratory ‘spin’ scores produced a total ‘spin’ score. Discrepancies in the total ‘spin’ score of >2 points between reviewers were resolved by consensus. If the difference between the reviewers’ scores was≤2 points, the mean value of the scores was included in the analysis. The inter-observer agreement for the ‘spin’ score was assessed in a sample of 10 press releases and news stories (5 industry and 5 non-industry) that were independent from the final study sample, being issued prior to 1 January 2005. In this analysis, the kappa coefficient was 0.64 (95% CI 0.59–0.84). The kappa coefficient is the proportion of agreement between two assessors, adjusted for chance agreement [16]. Values within the range 0.61–0.80 indicate substantial agreement [17].

Industry press release propagation

To assess the potential impact of the industry press releases, we sought news reports that referenced them. We searched the websites of six organizations (NewHope360, http://newhope360.com/; Nutritional Outlook, http://www.nutritionaloutlook.com/; Natural Products Insider, http://www.naturalproductsinsider.com/; Drug Store News, http://drugstorenews.com/; Nutraceuticals World, http://www.nutraceuticalsworld.com; Whole Food Magazine, http://www.wholefoodsmagazine.com/) that provide information and advice to manufacturers, marketers and retailers of dietary supplements, using search terms that included the sources of the industry press releases and each of the interventions studied in the source publications. From each site, we collated all news stories that directly referenced, or contained verbatim material from, any of the industry press releases in our sample set.

Statistics

The sample size of industry press releases was pragmatically determined by the number issued during the study period. Analyses were performed using Graph Pad Prism version 6.02 (http://www.graphpad.com). Between-group comparisons of continuous variables were made using Wilcoxon rank sum text. Analyses of categorical data were performed using Fisher’s exact test test or chi-squared test, as appropriate. Confidence intervals about medians were calculated using Graph Pad Prism version 6.02 and about percentages using Open Source Epidemiologic Statistics for Public Health (http://www.openepi.com), accessed November 2013. Since all comparisons were pre-planned no adjustment for multiplicity was performed. All tests were two tailed and P<0.05 was considered significant. Data are presented as median (95% CI) unless otherwise stated.

Results

Industry press releases and source publications

The final dataset contained 47 industry press releases (39 from the Council for Responsible Nutrition (CRN),; 3 from the Natural Products Association (NPA); 5 from the Alliance for Natural Health (ANH)). Some industry press releases referred to more than one source article, and some source articles generated press releases from more than one industry source. Consequently, the 47 industry press releases contained 53 responses to 46 source publications [18]–[63] (Table 1). Thirty-eight (83%) of the source articles were published in the seven most prestigious internal medicine journals, as judged by impact factor. The median (range) impact factor of the journals in which the source articles were published was 11 (4–51). Thirty-nine (85%) studies reported “hard” disease outcomes.

[Figure omitted. See PDF.]

Table 1. Press releases, news stories and source publications.

https://doi.org/10.1371/journal.pone.0101533.t001

Table S1 summarizes the types of studies reported by the source publications, according to study outcomes. Most of the studies (37/46, 80%) were randomized trials or meta-analyses of randomized trials, and most (28/46, 61%) reported no effect of the supplement investigated.

Non-industry press releases and news stories

Table 1 contains the sources of the non-industry press releases and news stories. One non-industry media document referred to two source articles. Consequently, there were 91 non-industry press releases and news stories, containing 92 responses to the 46 source publications. 55 (60%) of the non-industry press releases and news stories were from news agencies, 28 (31%) from the journal of publication, and 8 (9%) from the National Institutes of Health (NIH)/National Center for Complementary and Alternative Medicine (NCCAM).

‘Spin’ analyses

At least 1 item of ‘spin’ was present in all 53 industry media responses, compared to 51/92 (55%) non-industry media responses (P<0.001). The prevalence of hyping ‘spin’ did not differ between industry and non-industry media responses (16/53, 30%, vs 38/92, 41%; P = 0.21) but denigratory spin was more frequent in the industry media responses (40/53, 75% vs 21/92, 23%; P<0.001).

‘Spin’ scores differed by >2 points between reviewers for 19/145 (13%) of media responses, 13/53 industry and 6/92 non-industry. Figure 2a shows the total ‘spin’ scores in the industry and non-industry media responses. Considering all studies, the ‘spin’ score in the industry media responses (median 6, 95%CI 5.5–6.5) was higher than that in the non-industry media responses (0.5, 0.0–1.0; P<0.001). ‘Spin’ scores were higher in the industry media responses for studies that reported benefit, reported no benefit, or reported harm of the supplement (P<0.001 for each).

[Figure omitted. See PDF.]

Figure 2. Total (a), hyping (b) and denigratory (c) ‘spin’ scores of industry and non-industry media responses, according to outcomes of source publications.

Each point represents the ‘spin’ score of an individual media response. Where multiple scores of the same value occur, the number of overlapping ‘spin’ scores is indicated in parentheses to the right of the overlapping values. Bars represent the median and 95% CI.

https://doi.org/10.1371/journal.pone.0101533.g002

To investigate whether directional ‘spin’ was present, we analyzed the hyping and denigratory ‘spin’ scores according to the outcome of the study (Figures 2b–c). Considering all studies, hyping ‘spin’ scores were similar between industry and non-industry media responses (P = 0.74). For studies that reported benefit of supplements, hyping ‘spin’ scores were higher in industry media responses (3.3, 1.0–5.5) than in the non-industry media responses (0.5, 0–1.0; P<0.001). Hyping ‘spin’ scores were not different between industry and non-industry media responses for studies that reported no effect (P = 0.57) or harm (P = 0.06). Considering all studies, the denigratory ‘spin’ score was higher in industry media responses (6, 5.0–6.5) than non-industry media responses (0, 0–0; P<0.001). No denigratory ‘spin’ was present in industry media responses to studies reporting benefit of a supplement. Denigratory ‘spin’ scores were significantly higher in industry than non-industry media responses to publication of studies that found no effect (6.0, 5.0–7.0 vs 0, 0–0; P<0.001) or harm (6.0, 5.5–7.5 vs 0, 0–0.5; P<0.001) from a supplement. Removing responses from any of the non-industry sources (journals of publication, NIH, or news agencies) did not change the results of any of these analyses (data not shown).

The frequency of ‘spin’ techniques that were identified in at least 15% of media responses from either industry or non-industry sources is shown in Table S2.

Interpretation of press releases and news stories

There was no instance of assessor disagreement over the disposition of the media documents towards use of supplements. Industry media responses were more likely than non-industry media responses to support use of supplements. Considering all studies, the text of 51/53 (96%, 87–100) industry media responses was supportive of use of the supplement(s), in comparison to 25/92 (27%, 19–37) non-industry media responses (P<0.001) (Figure 3a). Industry and non-industry media responses were equally supportive of use of the supplement(s) in studies which reported benefit. For studies reporting no effect or harm of a supplement, the text of 97% (83–100) and 93% (66–100), respectively, of industry media responses was supportive of use of the supplement, while 16% (8–28) and 0% (0–17) of non-industry media responses were supportive of use (P<0.001 for each industry vs non-industry comparison). A similar pattern of results was apparent when the titles of media responses were assessed (Figure 3b).

[Figure omitted. See PDF.]

Figure 3. Disposition of industry and non-industry media responses towards use of supplements, according to outcome of source publications.

Data on the bars are the percentage of full text (a) and titles (b) of media responses that are supportive, non-supportive or neutral towards supplement use.

https://doi.org/10.1371/journal.pone.0101533.g003

Reporting of study characteristics and outcomes

Reporting of study design did not differ between industry and non-industry media responses, but industry media responses were less likely than non-industry media responses to report sample size (18/53, 34% vs 88/93, 96%; P<0.001) or to identify study outcomes (33/53, 62% vs 92/92, 100%; P<0.001) (Table 2). Each of these differences was more marked for studies that reported no effect or harm of a supplement. Study outcomes were identified in 8/8 (100%) industry media responses to studies reporting benefit of supplements, but only 19/31 (61%) and 6/14 (43%) of those issued in response to studies reporting no effect or harm, respectively. Study outcomes were reported using relative numbers in 9/53 (17%) of industry media responses and 39/92 (42%) of non-industry media responses (P = 0.002); outcomes were reported using absolute numbers in 0/53 (0%) of industry media responses and 25/92 (27%) of non-industry media responses (P<0.001). A numerical description of the study outcome was present in 7/8 (88%) industry media responses to studies reporting benefit of a supplement, but only 2/31 (6%) and 1/14 (7%) media responses to studies reporting no effect or harm, respectively.

[Figure omitted. See PDF.]

Table 2. Reporting of study characteristics and outcomes.

https://doi.org/10.1371/journal.pone.0101533.t002

Industry media responses were less likely than non-industry media responses to include quotes from study investigators (0/53, 0% vs 84/92, 91%; P<0.001) or other commentators (4/53, 8% vs 59/92, 64%; P<0.001), and more likely to include quotes from industry employees (53/53, 100% vs 10/92, 11%; P<0.001).

Propagation of industry press releases

On the websites of six organizations that service, inform and advise manufacturers, marketers and retailers of dietary supplements, we identified 148 news stories that directly referenced an industry press release (Table S3). The median (range) number of news stories that referenced an industry press release per source publication was 3 (0–10). Industry press releases generated in response to 42/46 (91%) of the source publications were referenced in at least one news story on at least one of the websites.

Discussion

Research on press releases issued by pharmaceutical companies and academic institutions has emphasized their propensity to hype research findings, by reporting positive preliminary data, omitting important study information, failing to discuss caveats and limitations, and exaggerating the importance of the results [13], [64]. We are unaware of research evaluating press releases from the supplements industry, or examining denigratory ‘spin’. Our analysis suggests that press releases issued by organizations which represent and promote the commercial interests of the manufacturers and retailers of supplemental medicines in response to the publication of clinical research findings about supplements contain more ‘spin’ than press releases and news stories from non-industry sources. Notably, the ‘spin’ in industry press releases strongly favoured use of supplements. More hyping ‘spin’ was present in industry press releases than non-industry media documents generated in response to the small number of studies reporting beneficial effects of supplements. The majority of studies (83%) that prompted industry press releases reported no benefit or harm of the supplement. In response to these studies, industry press releases were enriched for denigratory ‘spin’, and were almost unanimously (>90%) supportive of use of the supplement.

Industry press releases were also less likely than non-industry media documents to report key study characteristics such as sample size and study outcomes. This difference was apparent only for studies which reported no effect or harm of the supplement. Failure to identify and report study outcomes and provide estimates of effect size were common techniques by which industry press releases downplayed the outcomes of studies which failed to demonstrate benefit of supplements. Industry press releases never included interviews with the authors of the source publication, and rarely included opinions on the findings of the source publication from non-industry experts.

Press releases influence news stories [10], [11] and news stories influence health behaviors [65]–[67]. We found evidence for propagation of supplements industry press releases by organizations whose primary function is to inform and advise the manufacturers, marketers and consumers of dietary supplements. It is likely that these news stories influenced the attitudes of retailers and marketers of supplements, and thereby the behaviors of supplements consumers. The news stories may also have directly influenced consumer behaviour. Most supplements users do not take the agents on the advice of a health care professional [1], and many do not discuss supplement use with their doctor [4], [5], while information sources such as people other than health professionals, the internet, magazines, and news stories are influential [2], [5]. It is therefore likely that the propagation of the ‘spin’-enriched industry press releases contributes to the ongoing, and even burgeoning, enthusiasm for use of supplements in the face of accumulating evidence of their ineffectiveness [6], [58], [68] and, in some cases, harm [6], [38]. Continued use of interventions which are promoted as having health benefits despite evidence for lack of efficacy may discourage use of interventions for which evidence for efficacy is established [14] and can have adverse financial consequences for individual users [2].

Our study has limitations. Although our analyses were of press releases from three prominent supplements industry organizations, the results might not apply to press releases from other industry organizations. Media document content analysis and interpretation were subjective, but were performed independently by two reviewers, with a high level of agreement, and consensus to resolve disagreements. A blinded analysis would be ideal but, because of substantial differences in style and formatting among the sources of media documents, it was not possible to achieve without altering the documents so much that the original structure and tone would be lost. The control set of media documents included news stories and press releases, but sensitivity analyses found similar results after exclusion of the former documents. Analyses of responses to studies reporting benefits of supplements were limited by the small sample size.

Our results suggest that press releases issued by the supplements industry in response to clinical research on its products consistently include ‘spin’ that promotes the use of supplements, regardless of the research findings. Journalists, health practitioners and advisors, and consumers of supplemental medicines should therefore be sceptical of the content of press releases issued by organizations representing the supplements industry in response to clinical studies of supplements, and seek information from non-industry sources where the use of ‘spin’ is less common.

Supporting Information

[Figure omitted. See PDF.]

Table S1.

Design and outcome of studies that generated industry press releases.

https://doi.org/10.1371/journal.pone.0101533.s001

(DOCX)

Table S2.

Frequency of the most common ‘spin’ techniques observed in press releases or news stories.

https://doi.org/10.1371/journal.pone.0101533.s002

(DOCX)

Table S3.

Propagation of industry press releases by organizations that service manufacturers and retailers of supplements.

https://doi.org/10.1371/journal.pone.0101533.s003

(DOCX)

Text S1.

Data Abstraction Form 1 - Source Publications.

https://doi.org/10.1371/journal.pone.0101533.s004

(DOCX)

Text S2.

Data Abstraction Form 2 - Press Releases and New Stories.

https://doi.org/10.1371/journal.pone.0101533.s005

(DOCX)

Text S3.

‘Spin’ assessment.

https://doi.org/10.1371/journal.pone.0101533.s006

(DOCX)

Author Contributions

Conceived and designed the experiments: MW GG MB AG. Performed the experiments: MW AG. Analyzed the data: MW GG. Wrote the paper: MW GG MB AG.

Citation: Wang MTM, Gamble G, Bolland MJ, Grey A (2014) Press Releases Issued by Supplements Industry Organisations and Non-Industry Organisations in Response to Publication of Clinical Research Findings: A Case-Control Study. PLoS ONE 9(7): e101533. https://doi.org/10.1371/journal.pone.0101533

References

1. Bailey Rl, Gahche JJ, Miller PE, Thomas PR, Dwyer JT (2013) Why US adults use dietary supplements. JAMA Intern Med 173: 355–361.

2. Bacon CJ, Bolland MJ, Ames RW, Siu ATY, Mason BH, et al. (2011) Prevalent dietary supplement use in older New Zealand men. NZ Med J 124: 55–62.

3. Denison HJ, Jameson KA, Syddall HE, Dennison EM, Cooper C, et al. (2012) Patterns of dietary supplement use among older men and women in the UK: findings from the Hertfordshire Cohort Study. J Nutr Health Aging 16: 307–311.

4. Blendon RJ, Benson JM, Botta MD, Weldon KJ (2013) Users’ views of dietary supplements. Arch Intern Med 173: 74–75.

5. Albertazzi P, Steel SA, Clifford E, Bottazzi M (2002) Attitudes towards and use of dietary supplementation in a sample of postmenopausal women. Climacteric 5: 374–382.

6. Bjelakovic G, Nikolova D, Gluud C (2013) Antioxidant supplements to prevent mortality. JAMA 310: 1178–1179.

7. The DIPART Group (2010) Patient level pooled analysis of 68 500 patients from seven major vitamin D fracture trials in US and Europe. BMJ 340: b5463.

8. Klein EA, Thompson IM, Tangen CM, Crowley JJ, Lucia MS, et al. (2011) Vitamin E and the risk of prostate cancer. Journal of the American Medical Association 306: 1549–1556.

9. Grey A, Bolland M (2014) Clinical trial evidence and use of fish oil supplements. JAMA Intern Med 174: 460–462.

10. Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, et al. (2012) Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med 9: e1001308.

11. Schwartz LM, Woloshin S, Andrews A, Stukel TA (2012) Influence of medical journal press releases on the quality of associated newspaper coverage: retrospective cohort study. BMJ 344: d8164.

12. Boutron I, Dutton S, Ravaud P, Altman DG (2010) Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 303: 2058–2064.

13. Woloshin S, Schwartz L, Casella S, Kennedy A, Larson R (2009) Press releases by academic medical centers: not so academic? Ann Intern Med 150: 613–618.

14. Prasad V, Cifu A, Ioannidis JPA (2012) Reversals of established medical practices: evidence to abandon ship. JAMA 307: 37–38.

15. Tatsioni A, Bonitsis NG, Ioannidis JA (2007) Persistence of contradicted claims in the literature. JAMA 298: 2517–2526.

16. de Vet HC, Mokkink LB, Terwee CB, Hoekstra OS, Knol DL (2013) Clinicians are right not to like Cohen’s κ. BMJ 346: f2125.

17. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33: 159–174.

18. Lee IM, Cook NR, Gaziano JM, Gordon D, Ridker PM, et al. (2005) Vitamin E in the primary prevention of cardiovascular disease and cancer: the Women’s Health Study: a randomized controlled trial. JAMA 294: 56–65.

19. Lonn E, Bosch J, Yusuf S, Sheridan P, Pogue J, et al. (2005) Effects of long-term vitamin E supplementation on cardiovascular events and cancer: a randomized controlled trial. JAMA 293: 1338–1347.

20. Miller ER 3rd, Pastor-Barriuso R, Dalal D, Riemersma RA, Appel LJ, et al. (2005) Meta-analysis: high-dosage vitamin E supplementation may increase all-cause mortality. Ann Intern Med 142: 37–46.

21. Bent S, Kane C, Shinohara K, Neuhaus J, Hudes ES, et al. (2006) Saw palmetto for benign prostatic hyperplasia. N Engl J Med 354: 557–566.

22. Bonaa KH, Njolstad I, Ueland PM, Schirmer H, Tverdal A, et al. (2006) Homocysteine lowering and cardiovascular events after acute myocardial infarction. N Engl J Med 354: 1578–1588.

23. Clegg DO, Reda DJ, Harris CL, Klein MA, O’Dell JR, et al. (2006) Glucosamine, chondroitin sulfate, and the two in combination for painful knee osteoarthritis. N Engl J Med 354: 795–808.

24. Jackson RD, LaCroix AZ, Gass M, Wallace RB, Robbins J, et al. (2006) Calcium plus vitamin D supplementation and the risk of fractures. N Engl J Med 354: 669–683.

25. Lonn E, Yusuf S, Arnold MJ, Sheridan P, Pogue J, et al. (2006) Homocysteine lowering with folic acid and B vitamins in vascular disease. N Engl J Med 354: 1567–1577.

26. Prince RL, Devine A, Dhaliwal SS, Dick IM (2006) Effects of calcium supplementation on clinical fracture and bone structure: results of a 5-year, double-blind, placebo-controlled trial in elderly women. Arch Intern Med 166: 869–875.

27. Wactawski-Wende J, Kotchen JM, Anderson GL, Assaf AR, Brunner RL, et al. (2006) Calcium plus vitamin D supplementation and the risk of colorectal cancer. N Engl J Med 354: 684–696.

28. Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C (2007) Mortality in randomized trials of antioxidant supplements for primary and secondary prevention: systematic review and meta-analysis. JAMA 297: 842–857.

29. Cole BF, Baron JA, Sandler RS, Haile RW, Ahnen DJ, et al. (2007) Folic acid for the prevention of colorectal adenomas: a randomized clinical trial. JAMA 297: 2351–2359.

30. Lappe JM, Travers-Gustafson D, Davies KM, Recker RR, Heaney RP (2007) Vitamin D and calcium supplementation reduces cancer risk: results of a randomized trial. Am J Clin Nutr 85: 1586–1591.

31. Lin J, Manson JE, Lee IM, Cook NR, Buring JE, et al. (2007) Intakes of calcium and vitamin D and breast cancer risk in women. Arch Intern Med 167: 1050–1059.

32. Reichenbach S, Sterchi R, Scherer M, Trelle S, Burgi E, et al. (2007) Meta-analysis: chondroitin for osteoarthritis of the knee or hip. Ann Intern Med 146: 580–590.

33. Shah SA, Sander S, White CM, Rinaldi M, Coleman CI (2007) Evaluation of echinacea for the prevention and treatment of the common cold: a meta-analysis. Lancet Infect Dis 7: 473–480.

34. Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C (2008) Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Cochrane Database Syst Rev: CD007176.

35. Sawitzke AD, Shi H, Finco MF, Dunlop DD, Bingham CO 3rd, et al. (2008) The effect of glucosamine and/or chondroitin sulfate on the progression of knee osteoarthritis: a report from the Glucosamine/chondroitin Arthritis Intervention Trial. Arthritis Rheum 58: 3183–3191.

36. Sesso HD, Buring JE, Christen WG, Kurth T, Belanger C, et al. (2008) Vitamins E and C in the prevention of cardiovascular disease in men: the Physicians’ Health Study II randomized controlled trial. JAMA 300: 2123–2133.

37. Christen WG, Glynn RJ, Chew EY, Albert CM, Manson JE (2009) Folic acid, pyridoxine, and cyanocobalamin combination treatment and age-related macular degeneration in women: the Women’s Antioxidant and Folic Acid Cardiovascular Study. Arch Intern Med 169: 335–341.

38. Ebbing M, Bonaa KH, Nygard O, Arnesen E, Ueland PM, et al. (2009) Cancer incidence and mortality after treatment with folic acid and vitamin B12. JAMA 302: 2119–2126.

39. Ginde AA, Mansbach JM, Camargo CA Jr (2009) Association between serum 25-hydroxyvitamin D level and upper respiratory tract infection in the Third National Health and Nutrition Examination Survey. Arch Intern Med 169: 384–390.

40. Lin J, Cook NR, Albert C, Zaharris E, Gaziano JM, et al. (2009) Vitamins C and E and beta carotene supplementation and cancer risk: a randomized controlled trial. J Natl Cancer Inst 101: 14–23.

41. Neuhouser ML, Wassertheil-Smoller S, Thomson C, Aragaki A, Anderson GL, et al. (2009) Multivitamin use and risk of cancer and cardiovascular disease in the Women’s Health Initiative cohorts. Arch Intern Med 169: 294–304.

42. Park Y, Leitzmann MF, Subar AF, Hollenbeck A, Schatzkin A (2009) Dairy food, calcium, and risk of cancer in the NIH-AARP Diet and Health Study. Arch Intern Med 169: 391–401.

43. Snitz BE, O’Meara ES, Carlson MC, Arnold AM, Ives DG, et al. (2009) Ginkgo biloba for preventing cognitive decline in older adults: a randomized trial. JAMA 302: 2663–2670.

44. Armitage JM, Bowman L, Clarke RJ, Wallendszus K, Bulbulia R, et al. (2010) Effects of homocysteine-lowering with folic acid plus vitamin B12 vs placebo on mortality and major morbidity in myocardial infarction survivors: a randomized trial. JAMA 303: 2486–2494.

45. Bolland MJ, Avenell A, Baron JA, Grey A, MacLennan GS, et al. (2010) Effect of calcium supplements on risk of myocardial infarction and cardiovascular events: meta-analysis. BMJ 341: c3691.

46. Makrides M, Gibson RA, McPhee AJ, Yelland L, Quinlivan J, et al. (2010) Effect of DHA supplementation during pregnancy on maternal depression and neurodevelopment of young children: a randomized controlled trial. JAMA 304: 1675–1683.

47. Quinn JF, Raman R, Thomas RG, Yurko-Mauro K, Nelson EB, et al. (2010) Docosahexaenoic acid supplementation and cognitive decline in Alzheimer disease: a randomized trial. JAMA 304: 1903–1911.

48. Wandel S, Juni P, Tendal B, Nuesch E, Villiger PM, et al. (2010) Effects of glucosamine, chondroitin, or placebo in patients with osteoarthritis of hip or knee: network meta-analysis. BMJ 341: c4675.

49. Barry MJ, Meleth S, Lee JY, Kreder KJ, Avins AL, et al. (2011) Effect of increasing doses of saw palmetto extract on lower urinary tract symptoms: a randomized trial. JAMA 306: 1344–1351.

50. Klein EA, Thompson IM Jr, Tangen CM, Crowley JJ, Lucia MS, et al. (2011) Vitamin E and the risk of prostate cancer: the Selenium and Vitamin E Cancer Prevention Trial (SELECT). JAMA 306: 1549–1556.

51. Mursu J, Robien K, Harnack LJ, Park K, Jacobs DR Jr (2011) Dietary supplements and mortality rate in older women: the Iowa Women’s Health Study. Arch Intern Med 171: 1625–1633.

52. Bischoff-Ferrari HA, Willett WC, Orav EJ, Lips P, Meunier PJ, et al. (2012) A pooled analysis of vitamin D dose requirements for fracture prevention. N Engl J Med 367: 40–49.

53. Gaziano JM, Sesso HD, Christen WG, Bubes V, Smith JP, et al. (2012) Multivitamins in the prevention of cancer in men: the Physicians’ Health Study II randomized controlled trial. JAMA 308: 1871–1880.

54. Kramer H, Sempos C, Cao G, Luke A, Shoham D, et al. (2012) Mortality rates across 25-hydroxyvitamin D (25[OH]D) levels among adults with and without estimated glomerular filtration rate <60 ml/min/1.73 m2: the third national health and nutrition examination survey. PLoS One 7: e47458.

55. Kwak SM, Myung SK, Lee YJ, Seo HG (2012) Efficacy of omega-3 fatty acid supplements (eicosapentaenoic acid and docosahexaenoic acid) in the secondary prevention of cardiovascular disease: a meta-analysis of randomized, double-blind, placebo-controlled trials. Arch Intern Med 172: 686–694.

56. Li K, Kaaks R, Linseisen J, Rohrmann S (2012) Associations of dietary calcium intake and calcium supplementation with myocardial infarction and stroke risk and overall cardiovascular mortality in the Heidelberg cohort of the European Prospective Investigation into Cancer and Nutrition study (EPIC-Heidelberg). Heart 98: 920–925.

57. Murdoch DR, Slow S, Chambers ST, Jennings LC, Stewart AW, et al. (2012) Effect of vitamin D3 supplementation on upper respiratory tract infections in healthy adults: the VIDARIS randomized controlled trial. JAMA 308: 1333–1339.

58. Rizos EC, Ntzani EE, Bika E, Kostapanos MS, Elisaf MS (2012) Association between omega-3 fatty acid supplementation and risk of major cardiovascular disease events: a systematic review and meta-analysis. JAMA 308: 1024–1033.

59. Sesso HD, Christen WG, Bubes V, Smith JP, MacFadyen J, et al. (2012) Multivitamins in the prevention of cardiovascular disease in men: the Physicians’ Health Study II randomized controlled trial. JAMA 308: 1751–1760.

60. Chew EY (2013) Lutein/Zeaxanthin for the Treatment of Age-Related Cataract: AREDS2 Randomized Trial Report No. 4. JAMA Ophthalmol: 1–7.

61. Chew EY (2013) Lutein + zeaxanthin and omega-3 fatty acids for age-related macular degeneration: the Age-Related Eye Disease Study 2 (AREDS2) randomized clinical trial. JAMA 309: 2005–2015.

62. Michaelsson K, Melhus H, Warensjo Lemming E, Wolk A, Byberg L (2013) Long term calcium intake and rates of all cause and cardiovascular mortality: community based prospective longitudinal cohort study. BMJ 346: f228.

63. Xiao Q, Murphy RA, Houston DK, Harris TB, Chow WH, et al. (2013) Dietary and Supplemental Calcium Intake and Cardiovascular Disease Mortality: The National Institutes of Health-AARP Diet and Health Study. JAMA Intern Med 173: 639–646.

64. Kuriya B, Schneid EC, Bell CM (2008) Quality of pharmaceutical industry press releases based on original research. PLoS ONE 3: e2828.

65. Schwartz LM, Woloshin S (2004) The media matter: a call for straightforward medical reporting. Ann Intern Med 140: 226–228.

66. Grilli R, Ramsay C, Minozzi S (2002) Mass media interventions: effects on health services utilisation. Cochrane Database Syst Rev CD000389.

67. Haas JS, Kaplan CP, Gerstenberger EP, Kerlikowske K (2004) Changes in the use of postmenopausal hormone therapy after the publication of clinical trial results. Ann Intern Med 140: 184–188.

68. Clarke R, Halsey J, Lewington S, Armitage J, Manson JE, et al. (2010) Effects of lowering homocysteine levels with B vitamins on cardiovascular disease, cancer, and cause-specific mortality: Meta-analysis of 8 randomized trials involving 37 485 individuals. Arch Intern Med 170: 1622–1631.