Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes

Abstract

Understanding and improving the prognosis of a disease or health condition is a priority in clinical research and practice. In this article, the authors introduce a framework of four interrelated themes in prognosis research, describe the importance of the first of these themes (understanding future outcomes in relation to current diagnostic and treatment practices), and introduce recommendations for the field of prognosis research

Correspondence to: H Hemingway [email protected]

In clinical medicine, the term prognosis refers to the risk of future health outcomes in people with a given disease or health condition. Prognosis research is thus the investigation of the relations between future outcomes (endpoints) among people with a given baseline health state (startpoint) in order to improve health (see supplementary figure on bmj.com). The study of prognosis has never been more important, as globally more people are living with one or more disease or health impairing condition than at any previous time.¹ For this reason, governments across the world are increasing their interest in the outcomes of healthcare currently provided for people with disease.² Similarly, research funders and researchers are increasingly focused on translating new interventions and technologies from the laboratory to clinical practice and then healthcare policy in order to establish and implement new standards of high quality care and improve patient outcomes.

Prognosis research findings should thus be integral to clinical decision making, healthcare policy, and discovering and evaluating new approaches to patient management. However, there is a concerning gap between the potential and actual impact of prognosis research on health. Prognosis research studies too often fall a long way short of the high standards required in other fields, such as therapeutic trials and genetic epidemiology.

In the PROGnosis RESearch Strategy (PROGRESS) series (www.progress-partnership.org), we propose a framework of four distinct but inter-related prognosis research themes:

• (1) The course of health related conditions in the context of the nature and quality of current care (fundamental prognosis research)

• (2) Specific factors (such as biomarkers) that are associated with prognosis (prognostic factor research)³

• (3) The development, validation, and impact of statistical models that predict individual risk of a future outcome (prognostic model research)⁴

• (4) The use of prognostic information to help tailor treatment decisions to an individual or group of individuals with similar characteristics (stratified medicine research).⁵

Figure 1 illustrates these four prognosis research areas for women with breast cancer (startpoint) and the endpoint of death or disease-free survival. Part (a) shows country variations in age adjusted, five year survival (fundamental prognosis research)⁶; part (b) shows survival curves according to the value of extracellular domain of human epidermal growth factor receptor 2 (HER2 ECD), which is identified to be prognostic of disease outcome (prognostic factor research)⁷; part (c) shows the use of multiple clinical variables within a statistical model to estimate individual risk of a particular endpoint (prognostic model research)⁸; and part (d) shows why a positive oestrogen receptor status is used to identify those who will benefit from tamoxifen therapy (stratified medicine research).⁹

The overarching aim of the PROGRESS series is to explain how each of these four prognosis research themes provides important evidence that can be used at multiple (translational) pathways toward improving clinical outcomes—from the discovery of new interventions, through to their evaluation and implementation in the clinical management of individual patients, and to examining the impact of interventions and healthcare policies on patient outcomes. This contrasts with previous reviews of prognosis research which consider impact at one end of the translational spectrum (such as clinical decision making) or on just one type of prognosis question (such as prognostic models¹⁰). Whereas previous reviews focus on one specific disease area (such as cancer),^{11 12} we include examples from cancer, cardiovascular disease, musculoskeletal disorders, trauma, and other conditions. Our series describes the current challenges and opportunities in the field and makes recommendations for necessary improvements to move toward a clearer map for prognosis research that ultimately improves patient outcomes (summarised in supplementary table 1 on bmj.com).

An important place to start is with research that aims to examine the outcomes of a disease or health condition in the context of current clinical practice, and this we term fundamental prognosis research. In this first article we consider what this entails, explain its importance in pathways toward improving patient outcomes, and outline a set of recommendations with the aim of improving the quality and impact across all of the inter-related themes in prognosis research and which will be expanded in the other articles in our series.

What is fundamental prognosis research?

Before carrying out research into novel prognostic factors, prognostic models, or stratified medicine it is necessary to carry out research describing and explaining future outcomes in people with a disease or health condition in relation to current diagnostic and treatment practices. There is a close relation between the questions “What is the prognosis of people with this condition?” and “What are the outcomes of the care which people receive for this condition?” In order to improve the quality of healthcare, evidence is required on how the specific patterns of care received (such as investigation, treatment, support), and their variations (such as underuse, overuse, misuse) have an impact on future endpoints.¹³ Such research has a broad remit. It spans, for example, investigations into societal influences (inequitable variations in care and outcome among older people, women, the socially disadvantaged, and ethnic minorities), patient safety,^{14 15} unanticipated harms and benefits from treatments, and screening research. Prognosis in the absence of care—which is sometimes termed natural history—is an important parameter for judging the potential impact of screening for asymptomatic disease (such as mammography for breast cancer), as well as for case detection of symptomatic undiagnosed or unpresented conditions such as back pain or angina.^{16 17}

These relations may be expressed as an absolute risk (or rate) of one or more type of endpoint among groups of people who share demographic and clinical characteristics; some refer to this as an average prognosis in a particular group of interest, or as a baseline risk. Here the research provides initial answers to the question “What is the prognosis of people with a given disease?” For example, on average about 15% of people aged 65 years or older, admitted in 2006 in the US died within 30 days of admission to hospital with a heart attack compared with an average of 19% in 1995.¹⁸ Such a change in the average mortality rate is illustrated in figure 2 . This shows the decreasing prognostic burden of heart attack and motivates inquiry into new approaches to understand and reduce this risk further. This clinical scenario also exemplifies that “the prognosis” of a disease or condition is a somewhat misleading expression: what is observed is prognosis of people in particular clinical contexts, defined by current clinical approaches in diagnosing, characterising, and managing patients with a symptom or disease.

Such prognosis research is also concerned with describing and understanding the variations around the average course.^{19 20} These variations may occur between individual patients or between patients clustered, for example, within surgeons, hospitals, or regions. The acute myocardial infarction example above demonstrates striking variations between hospitals in prognosis, and similar variations are seen in traumatic brain injury and other conditions.^{18 21} Indeed, for most hospitals the national average is a poor guide to the mortality of their patients (fig 2 ).

Stephen J Gould, the evolutionary biologist, having survived 20 years after being told the median survival of his abdominal mesothelioma was eight months, famously remarked, “the median isn’t the message.”²² Describing and explaining the sources of variability in prognosis is a theme throughout our PROGRESS framework.^{3 4 5} Fundamental prognosis research may help explain Gould’s long survival in terms of the demographic and clinical context (for example, his high educational status and the quality of care received), whereas research into emerging prognostic factors may examine psychological, behavioural, or biomarker factors associated with improved outcome (see paper 2 in our series³), or the extent to which his survival was predictable from statistical models of individual risk prediction (paper 3 in our series⁴), or whether particular treatments had a larger beneficial effect for him than for others (paper 4 in our series⁵).

Importance of fundamental prognosis research in the pathways toward improved health outcomes

Healthcare professionals, people with a disease or health condition, funders, and policy makers require valid, reliable evidence about the outcomes of diseases and health conditions in order to make decisions. Here we review the potential impact of such evidence across translational pathways in healthcare, starting from the applied, healthcare delivery end (far right of pathways schema shown at bottom of figs 2 , 3 , and 4 ) and working back to discovery and new approaches (far left of schema).

Importance for public health policy

Public health policy makers need estimates of average prognosis to model the population burden of diseases and assess the relative contribution of healthcare delivery among those with disease (secondary prevention) and without disease (primary prevention). For example, the public health objective of reducing overall coronary heart disease mortality (a conflation of incidence of non-fatal coronary disease and subsequent death) has been helped by modelling the impact of population interventions aimed at early detection and primary and secondary prevention.^{23 24 25} Such models use an average prognosis of heart attack survival from the date of diagnosis among age and sex strata to attribute quality adjusted life years (and health service costs of managing the disease) which would be saved with successful prevention.

By contrast with the improvements over time in the prognosis of coronary disease, for people with low back pain there is little evidence that the average prognosis (based on symptom relief ^{26 27}) has changed over the past 20 years, nor does it differ between countries with different healthcare systems.²⁸ This suggests that healthcare itself is not a major influence on average symptomatic outcome in people with back pain. However, when considering the outcome of sickness absence, there are dramatic variations over time and between countries—suggesting the importance of the broader public health context of working patterns and benefit payments for chronic illness.²⁹

Importance for comparative effectiveness and health services research

Insights into health and healthcare policy may come from comparing the prognosis of specific conditions over time and place in order to assess the comparative effectiveness of systems of care.^{30 31} For example, figure 1 shows that the five year survival from breast cancer in 2000-03 varies widely from country to country (from about 70% in the Czech Republic to 90% in Iceland). The UK seems to have worse cancer survival than most other European countries,³² and the latter have worse survival for some cancers than the US. Such international comparisons of average prognosis provide a motivation for researchers to uncover explanations and for healthcare policy makers to improve the quality of care and deliver better health outcomes.² Policy makers seeking to improve national cancer outcomes may consider a range of interventions, including: early detection (such as mammography screening), population-wide guidance (such as encouraging self examination),^{33 34} centralisation of services, and systematic implementation of cost effective therapies. Ecological comparisons of country-level factors (such as smoking prevalence or number of specialists per capita population) can be related to outcomes. Such research may generate hypotheses for prognostic factor research (see paper 2 in our series³) as well as helping to formulate service and policy development.

Fundamental prognosis research is vital in addressing the “second gap” in translation,³⁵ in which evidence from randomised trials of effective treatments may fail to be implemented in usual clinical practice (far right of translational pathway toward improved clinical outcomes). For example, the between hospital variations in outcome from acute myocardial infarction (fig 2 ) may, in part, stem from differing use of evidence based therapies. These findings have profound implications for healthcare policy. It demonstrates a “normal distribution” of mortality between hospitals; over time the whole distribution of hospital mortality improves and shifts to the left and the variation between hospitals in outcomes narrows. The policy implication is that improvements in the quality of care in the population of all hospitals may have contributed to the observed shift in the average prognosis. Thus the evidence did not support a contrasting policy alternative of focusing on the identification of, and remedial action in, outlying poor performers.³⁶ Here prognosis research is contributing evidence about health services and is managing knowledge generated from electronic health records. Such evidence³⁵ informs policy choices which are themselves highly unlikely to be subjected to randomised trials.³⁷

Importance for health technology assessment of imaging and other tests

A key target for translational research is the development of new clinical imaging and molecular markers which may identify patient phenotypes in such a way as to lead to improved outcomes. Such new technologies may change the spectrum of diagnosed disease, and the question is whether prognosis is the same as with the use of standard tests and whether the balance of benefit and harm of treatment remains the same. For example, for decades exercise electrocardiography has been used in the characterisation of patients with stable chest pain, and recent guidelines recommend the use of an emerging technology, non-invasive computed tomographic coronary angiography, in some patients.³⁸ Since event powered randomised trials of imaging remain rare, fundamental prognosis research provides an important method of health technology assessment.³⁹

Importance for trials and decision models

Estimates of average prognosis are also crucial for the rationale, design, interpretation, and impact modelling of trials of an intervention to improve prognosis. For example, prognosis research among people with angina shows that 50% of people with existing therapies have recurrent or persistent symptoms,⁴⁰ suggesting the need for trials of new interventions. Reliable estimates of prognosis inform the estimates of likely accrual of endpoints in the trial arms (such as expected proportion experiencing an event by a particular time), and hence facilitate statistical sample size calculations. They also contribute to the interpretation in terms of generalisability of clinical trial results, as one can compare the average prognosis of patients in the trial without treatment with the average prognosis in particular populations.

Importantly, in order to translate relative treatment effects (such as relative risks or hazard ratios) back to the absolute scale, one needs to know the average prognosis (baseline risk) in the untreated group. One can then talk in terms of the reduction in probability of a poor outcome (risk difference), which leads to clinically informative measures such as the number needed to treat in order to save one patient from a particular poor outcome. Absolute effects are used within decision models and cost effectiveness analyses, which are highly influential to decision makers such as the National Institute of Health and Clinical Excellence (NICE). Such models combine parameters of average prognosis along with estimates of treatment effects and costs. Conclusions from these models are often particularly sensitive to the accuracy of the data on average prognosis among those without the specific treatment of interest.

Importance for new approaches, mechanisms, and targets for trials

Fundamental prognosis research may provide insights beyond evaluating the status quo of clinical care. Estimating the prospective associations between two diseases has led to startling discoveries that have stimulated the development of new interventions and new clinical trials that have ultimately changed clinical practice. For example, few foresaw that a prognostic consequence of Helicobacter pylori infection was peptic ulcer before the Nobel prize winning work that established the link and subsequent antibiotic trials.⁴¹ Importantly, the outcomes of uncommon conditions may give insights into disease mechanisms of common conditions. For example, the increased risk of coronary outcomes among people with familial hypercholesterolaemia focused interest on the low density lipoprotein cholesterol pathways which are important in coronary disease experienced by people without this genetic disorder and contributed to the development of lipid lowering therapy.

Taking a broad view of prognostic outcomes may generate new knowledge at the start of translational pathways with (as yet) unknown implications for developing new interventions. Consider the example of following up people with Parkinson’s disease. The risk of cancer is not an endpoint that would conventionally be considered. However, a meta-analysis found that the risk of cancer was significantly reduced compared with people without Parkinson’s disease (fig 3 ).⁴² This raises the question whether specific characteristics of Parkinson’s disease that explain this apparent protective effect can be identified, and whether this might lead to new intervention targets. There are probably many prognostic associations between two or more diseases that have yet to be uncovered. Some have proposed that approaches using all available clinical data (so called phenome-wide scans), agnostic to any prior theories about mechanism, might identify new associations between conditions.⁴³

Importance for overcoming the limitations of diagnosis

The understanding of future outcome risk (prognosis) may be a more useful way of formulating clinical problems than pursuing diagnosis for several reasons. First, subjectively reported illness such as mental health problems and pain syndromes is often managed more with prognostic than diagnostic labels.⁴⁴ For example, a physician may reasonably say to a person presenting with back pain, “I do not know what is wrong, but I do know that this is the sort of back pain that is very likely to get better quickly.” Evidence from prognosis research has helped to redefine low back pain. Spinal radiography and magnetic resonance imaging contribute little to understanding the average prognosis of most back pain,^{16 45} but the duration of symptoms at presentation in primary care is strongly related to outcome. Figure 4 shows that the chance of reduced disability at one year is about 70% in those with a shorter duration (<3 years) of symptoms at presentation versus 40% in those with a longer duration.^{26 29 46} Clinical practice guideline recommendations use symptom duration to guide management decisions.⁴⁷ Symptom duration is associated with clinical outcome and is thus a prognostic factor (see paper 2 in our series³), which has resulted in it being a standard component of the clinical evaluation of back pain.

Second, fundamental prognosis research can take a holistic view of all comorbidities that a person experiences, whereas diagnosis implies a focus on a single organ system or pathology. The prognosis of some cancers, traumatic brain injury, and back pain are importantly influenced by conditions not related to the tumour, brain, and spine respectively. Third, diagnosis implies a dichotomy (case v not at a single point in time), which may be a misleading basis for clinical decision making. For example, in many countries the decision to lower blood cholesterol is not based on a diagnosis of hypercholesterolaemia but on thresholds of continuous risk, determined by age, sex, smoking, blood pressure, and lipids (see paper 3 in our series⁴). Such observations have led to the radical proposition that the dichotomous, cross sectional snapshot of diagnostic practice may become redundant, as clinicians increasingly have access to continuous measures of future risk.^{48 49}

Importance for discovering new diseases

Fundamental prognosis research drives definitions of the diseases for which interventions are sought.⁵⁰ Such research helps define our current view of what distinct clinical conditions exist and what role new clinical tests might have in changing our classification of disease entities (nosology). The question “what is the prognosis of this condition?” is intimately related to the question “what is this condition?” For example, the entity of non-fatal myocardial infarction was identified only after many decades of clinical prognostic observation that symptoms of chest pain may precede death, replacing the view that the disease of myocardial infarction was inevitably and instantly fatal. More recently, prognosis research has helped to redefine non-fatal acute myocardial infarctions⁵¹ based on the presence or absence of ST elevation, a predictor of differential response to therapy, and serum troponin measurement. Figure 5 shows that examination of survival patterns differentiates clinical phenotypes among people admitted with suspected non-fatal myocardial infarction. An example of a newly recognised genetic disorder discovered through prognostic observation is Brugada syndrome in which an ST elevation pattern on resting electrocardiogram is associated with sudden death.⁵²

Recommendations for improving the quality and impact of prognosis research

For each of the four themes of prognosis research to achieve its potential for improving clinical outcomes, important challenges need to be addressed and opportunities seized in prognosis research as a whole. The research community needs to address serious flaws in the design, conduct, and reporting of prognosis studies and to recognise the clinical value of reliable prognostic evidence. In the PROGRESS series we thus make recommendations for progress in the field, and these are summarised in supplementary table 1 on bmj.com. Here we introduce recommendations that cut across the different research themes. In papers 2–4 in the PROGRESS series,^{3 4 5} we discuss the other recommendations from supplementary table 1. These recommendations add to, and further specify, those which we have previously made in the BMJ.⁵³

Fuelling changes in medicine and healthcare

As shown in the examples above, improvements in electronic health records, clinical imaging, and “omic” technologies (genotyping and phenotyping) are beginning to challenge current disease taxonomy, the focus of much healthcare policy on process (rather than clinical outcomes), and the clinical preoccupation with diagnosis (rather than risk). There should be a formative shift in clinical practice, healthcare policy, and translational research based on evidence from prognosis research—that is, the prospective relationships between the phenotypic, genomic, and environmental assessment of people with a given startpoint and subsequent endpoints (recommendation 1 in supplementary table 1). Over their life course, individuals develop multiple diseases (both distinct and related) that often do not respect the current organisation of medical research or practice. There should be new programmes of prognosis research that bridge multiple clinical specialties, health systems, pathological mechanisms, and biological systems and that put the whole patient across his or her “journey” as the central unit of concern (recommendation 2).

Electronic health records

The scope and impact of prognosis research and electronic health records research (in primary and secondary care, and in disease and procedure registries) are intimately related. There is increasing availability of electronic health records in primary⁵⁴ and secondary care, and disease and procedure registries. Particularly where such sources can be linked,⁵⁵ there is the possibility of examining the “patient journey” with repeated measures of risk and care in larger populations than are feasible with bespoke, investigator led studies. Population coverage, data quality, and the extent of blood, imaging, and other diagnostic data are all improving. But concerted efforts are required to harmonise data on startpoints, endpoints, and populations of interest in order to make temporal and international comparisons in prognosis. There should be new programmes of methodological and empirical prognosis research exploiting electronic health records to define, phenotype, and follow up people with different health related conditions (recommendation 3).

Visibility of the field

Prognosis research is currently fragmented and not visible as a distinct entity. Prognosis research should be recognised as a field of inquiry important in translational research and intrinsic to the practice of clinical medicine and development of healthcare policy (recommendation 4). Efforts should be made to establish prognosis research as a distinct branch of knowledge, with a set of scientific methods aimed at understanding and improving health. Evidence about prognosis is somewhat neglected; such as in medical textbooks, where the focus is on the effectiveness of therapies, with only brief details given on average prognosis,⁵⁶ sometimes as if therapies can be divorced from the context of clinical care.^{57 58}

Fundamental prognosis research should compare the prognosis of clinical cohorts with that of the healthy population (recommendation 5). Relative survival methods are commonly applied in cancer, but less often in other disease areas. Relative survival methods model the survival probability of people with a condition relative to the expected survival without the condition (obtained from national population life tables stratified by age, sex, calender year, and other covariates). By comparing the observed and expected survival, one can estimate the added risk of mortality due to having the condition rather than not having it (that is, measure how prognosis is modified by onset of a disease). Such methods help prognosis research prioritise which clinical cohorts require the most attention and most translational research (that is, identify those cohorts whose prognosis is most changed by disease onset).

The situation for cancer, where estimates of survival are readily available (such as Surveillance Epidemiology and End Results, SEER⁵⁹) is exceptional. Knowledge management in prognosis seems somewhat chaotic in generation, dissemination, and accessibility. Difficulties in identifying and accessing information about prognosis, and evidence from prognosis research studies, hamper efforts to inform patients and evaluate the impact of translational efforts to improve outcomes. Evidence from prognosis research and information about prognosis should be systematically collated, made easily accessible, and updated (recommendation 6).

Teaching and training

Undergraduate and postgraduate training do not currently provide instruction in how to generate or use evidence from prognosis research. All healthcare professionals should be trained in the generation and use of prognosis research evidence; there should be an expansion of training and education opportunities for those interested in methodological aspects of prognosis research (recommendation 7).

Patient and public involvement

Questions of prognosis are among the most important to patients, but the level of patient and public involvement in prognosis research is low. Patient reported outcomes are important to clinical decision and policy making but are understudied. For example, people with angina might reasonably ask “will my symptoms get better?” yet a recent systematic review of 83 studies found none that reported symptomatic status as an endpoint (favouring acute coronary events instead).⁶⁰ Symptom status is acknowledged as a major determinant of the clinical decision to recommend revascularisation.⁶¹ Prognosis research using person focused endpoints may yield unanticipated results. For example, people with rheumatoid arthritis may care more about fatigue than about the joint pain, on which doctors tend to focus.⁶² Patients and the wider public should be more engaged in the goals and value of prognosis research, appropriate use of their clinical data, and better integration of patient reported outcome measures (recommendation 8).

Conclusion

In this first article in the PROGRESS series, we have introduced a framework of four themes in prognosis research, and outlined the importance of initial, fundamental prognosis research. This first theme is central to the practice of medicine; from basic understanding of the categories we choose to call disease through to understanding how variations in healthcare influence the risk of endpoints. As such, it has a broad array of uses for policy makers, patients, and clinical decision making and should be considered a core component of prognosis research. To maximise the impact of each interrelated theme of prognosis research,^{3 4 5} we have begun outlining a set of recommendations to enhance the prognosis field, including better use of electronic health records, greater training and public involvement, and a wider appreciation of the clinical value of prognosis research findings.

Summary points

• The PROGRESS series (www.progress-partnership.org) sets out a framework of four interlinked prognosis research themes and provides examples from several disease fields to show why evidence from prognosis research is crucial to inform all points in the translation of biomedical and health related research into better patient outcomes. Recommendations are made in each of the four papers to improve current research standards

• What is prognosis research? Prognosis research seeks to understand and improve future outcomes in people with a given disease or health condition. However, there is increasing evidence that prognosis research standards need to be improved

• Why is prognosis research important? More people now live with disease and conditions that impair health than at any other time in history; prognosis research provides crucial evidence for translating findings from the laboratory to humans, and from clinical research to clinical practice

• This first article introduces the framework of four interlinked prognosis research themes and then focuses on the first of the themes—fundamental prognosis research, studies that aim to describe and explain future outcomes in relation to current diagnostic and treatment practices, often in relation to quality of care

• Fundamental prognosis research provides evidence informing healthcare and public health policy, the design and interpretation of randomised trials, and the impact of diagnostic tests on future outcome. It can inform new definitions of disease, may identify unanticipated benefits or harms of interventions, and clarify where new interventions are required to improve prognosis

• The other papers in the series are:

  • PROGRESS 2: PLoS Med 2013, doi:10.1371.journal/pmed.1001380

  • PROGRESS 3: PLoS Med 2013, doi:10.1371.journal/pmed.1001381

  • PROGRESS 4: BMJ 2013, doi:10.1136/bmj.e5793

Cite this as: BMJ 2013;346:e5595

¹ Mathers CD, Loncar D. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med 2006; 3: e442.

² Department of Health. White paper. Equity and excellence: liberating the NHS. Stationery Office, 2010.

³ Riley RD, Hayden JA, Steyerberg EW, Moons KGM, Abrams K, Kyzas PA, et al. Prognosis research strategy (PROGRESS) 2: Prognostic factor research. PLoS Med 2013, doi: 10.1371.journal/pmed.1001380.

⁴ Steyerberg EW, Moons KGM, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis research strategy (PROGRESS) 3: Prognostic model research. PLoS Med 2013, doi: 10.1371.journal/pmed.1001381.

⁵ Hingorani AD, van der Windt DA, Riely RD, Abrams K, Moons KGM, Steyerberg EW, et al. Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ 2013; 346: e5793.

⁶ Cancer Research UK. CancerStats: cancer statistics for the UK. Cancer Research UK, 2009. (Original data source: Verdecchia et al.Recent cancer survival in Europe: a 2000-02 period analysis of EUROCARE-4 data. Lancet Oncology 2007; 8: 784 -96.)

⁷ Tsai H, Chen S, Chien H, Jan Y, Chao T, Chen M, et al. Relationships between serum HER2 ECD, TIMP-1 and clinical outcomes in Taiwanese breast cancer. World J Surg Oncol 2012; 10: 42.

⁸ Adjuvant! Online: Decision making tools for health care professionals. www.adjuvantonline.com/index.jsp.

⁹ Early Breast Cancer Trialists Collaborative Group. Tamoxifen for early breast cancer: an overview of the randomised trials. Lancet 1998; 351: 1451 -67.

¹⁰ Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ 2009; 338: b375.

¹¹ Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MS, et al. Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation 2009; 119: 2408 -16.

¹² McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM, et al. Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 2005; 97: 1180 -4.

¹³ Krumholz HM. Outcomes research: generating evidence for best practice and policies. Circulation 2008; 118: 309 -18.

¹⁴ Nissen H, Rosnes JT, Brendehaug J, Kleiberg GH. Safety evaluation of sous vide-processed ready meals. Lett Appl Microbiol 2002; 35: 433 -8.

¹⁵ Sipahi I, Debanne SM, Rowland DY, Simon DI, Fang JC. Angiotensin-receptor blockade and risk of cancer: meta-analysis of randomised controlled trials. Lancet Oncol 2010; 11: 627 -36.

¹⁶ Chou R, Shekelle P. Will this patient develop persistent disabling low back pain? JAMA 2010; 303: 1295 -302.

¹⁷ Hemingway H, Shipley M, Britton A, Page M, Macfarlane P, Marmot M. Prognosis of angina with and without a diagnosis: 11 year follow up in the Whitehall II prospective cohort study. BMJ 2003; 327: 895.

¹⁸ Krumholz HM, Wang Y, Chen J, Drye EE, Spertus JA, Ross JS, et al. Reduction in acute myocardial infarction mortality in the United States: risk-standardized mortality rates from 1995-2006. JAMA 2009; 302: 767 -73.

¹⁹ Henderson R, Keiding N. Individual survival time prediction using statistical models. J Med Ethics 2005; 31: 703 -6.

²⁰ Royston P, Parmar MK, Altman DG. Visualizing length of survival in time-to-event studies: a complement to Kaplan-Meier plots. J Natl Cancer Inst 2008; 100: 92 -7.

²¹ Lingsma HF, Roozenbeek B, Li B, Lu J, Weir J, Butcher I, et al. Large between-center differences in outcome after moderate and severe traumatic brain injury in the international mission on prognosis and clinical trial design in traumatic brain injury (IMPACT) study. Neurosurgery 2011; 68: 601 -7.

²² CancerGuide: Statistics. The median isn’t the message. http://cancerguide.org/median_not_msg.html (updated 2002).

²³ Ford ES, Ajani UA, Croft JB, Critchley JA, Labarthe DR, Kottke TE, et al. Explaining the decrease in US deaths from coronary disease, 1980-2000. N Engl J Med 2007; 356: 2388 -98.

²⁴ Hunink MG, Goldman L, Tosteson AN, Mittleman MA, Goldman PA, Williams LW, et al. The recent decline in mortality from coronary heart disease, 1980-1990. The effect of secular trends in risk factors and treatment. JAMA 1997; 277: 535 -42.

²⁵ Weinstein MC, Coxson PG, Williams LW, Pass TM, Stason WB, Goldman L. Forecasting coronary heart disease incidence, mortality, and cost: the Coronary Heart Disease Policy Model. Am J Public Health 1987; 77: 1417 -26.

²⁶ Croft PR, Macfarlane GJ, Papageorgiou AC, Thomas E, Silman AJ. Outcome of low back pain in general practice: a prospective study. BMJ 1998; 316: 1356 -9.

²⁷ Von KM, Deyo RA, Cherkin D, Barlow W. Back pain in primary care. Outcomes at 1 year. Spine 1993; 18: 855 -62.

²⁸ Costa LC, Maher CG, McAuley JH, Hancock MJ, Herbert RD, Refshauge KM, et al. Prognosis for patients with chronic low back pain: inception cohort study. BMJ 2009; 339: b3829.

²⁹ Hestbaek L, Leboeuf-Yde C, Manniche C. Low back pain: what is the long-term course? A review of studies of general patient populations. Eur Spine J 2003; 12: 149 -65.

³⁰ De Silva MJ, Roberts I, Perel P, Edwards P, Kenward MG, Fernandes J, et al. Patient outcome after traumatic brain injury in high-, middle- and low-income countries: analysis of data on 8927 patients in 46 countries. Int J Epidemiol 2009; 38: 452 -8.

³¹ Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med 2009; 151: 203 -5.

³² Coleman MP, Quaresma M, Berrino F, Lutz JM, De AR, Capocaccia R, et al. Cancer survival in five continents: a worldwide population-based study (CONCORD). Lancet Oncol 2008; 9: 730 -56.

³³ Buchbinder R, Jolley D, Wyatt M. Population based intervention to change back pain beliefs and disability: three part evaluation. BMJ 2001; 322: 1516 -20.

³⁴ Buchbinder R, Jolley D. Population based intervention to change back pain beliefs: three year follow up population survey. BMJ 2004; 328: 321.

³⁵ Sung NS, Crowley WF Jr, Genel M, Salber P, Sandy L, Sherwood LM, et al. Central challenges facing the national clinical research enterprise. JAMA 2003; 289: 1278 -87.

³⁶ Rose G. The strategy of preventive medicine. Oxford University Press, 1993.

³⁷ Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 2007; 297: 278 -85.

³⁸ National Institute for Health and Clinical Excellence. Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin (clinical guideline 95). www.nice.org.uk/guidance/CG95 (updated 2010).

³⁹ Min JK, Shaw LJ, Devereux RB, Okin PM, Weinsaft JW, Russo DJ, et al. Prognostic value of multidetector coronary computed tomographic angiography for prediction of all-cause mortality. J Am Coll Cardiol 2007; 50: 1161 -70.

⁴⁰ Griffin SC, Barber JA, Manca A, Sculpher MJ, Thompson SG, Buxton MJ, et al. Cost effectiveness of clinically appropriate decisions on alternative treatments for angina pectoris: prospective observational study. BMJ 2007; 334: 624.

⁴¹ Malfertheiner P, Chan FK, McColl KE. Peptic ulcer disease. Lancet 2009; 374: 1449 -61.

⁴² Bajaj A, Driver JA, Schernhammer ES. Parkinson’s disease and cancer risk: a systematic review and meta-analysis. Cancer Causes Control 2010; 21: 697 -707.

⁴³ Hanauer DA, Rhodes DR, Chinnaiyan AM. Exploring clinical associations using ‘-omics’ based enrichment analyses. PLoS One 2009; 4: e5203.

⁴⁴ Kroenke K, Harris L. Symptoms research: a fertile field. Ann Intern Med 2001; 134 (9 Pt 2): 801 -2.

⁴⁵ Von Korff M, Miglioretti DL. A prognostic approach to defining chronic pain. Pain 2005; 117: 304 -13.

⁴⁶ Dunn KM, Croft PR. The importance of symptom duration in determining prognosis. Pain 2006; 121: 126 -32.

⁴⁷ O’Neill CJ, Spence A, Logan B, Suliburk JW, Soon PS, Learoyd DL, et al. Adrenal incidentalomas: risk of adrenocortical carcinoma and clinical outcomes. J Surg Oncol 2010; 102: 450 -3.

⁴⁸ Wiesemann C. The significance of prognosis for a theory of medical practice. Theor Med Bioeth 1998; 19: 253 -61.

⁴⁹ Vickers AJ, Basch E, Kattan MW. Against diagnosis. Ann Intern Med 2008; 149: 200 -3.

⁵⁰ Scadding JG. Diagnosis: the clinician and the computer. Lancet 1967; 2: 877 -82.

⁵¹ Thygesen K, Alpert JS, White HD, Jaffe AS, Apple FS, Galvani M, et al. Universal definition of myocardial infarction. Circulation 2007; 116: 2634 -53.

⁵² Probst V, Veltmann C, Eckardt L, Meregalli PG, Gaita F, Tan HL, et al. Long-term prognosis of patients diagnosed with Brugada syndrome: results from the FINGER Brugada Syndrome Registry. Circulation 2010; 121: 635 -43.

⁵³ Hemingway H, Riley RD, Altman DG. Ten steps towards improving prognosis research. BMJ 2009; 339: b4184.

⁵⁴ General Practice Research Database. www.gprd.com/home/default.asp (updated 2011).

⁵⁵ Jernberg T, Attebring MF, Hambraeus K, Ivert T, James S, Jeppsson A, et al. The Swedish Web-system for enhancement and development of evidence-based care in heart disease evaluated according to recommended therapies (SWEDEHEART). Heart 2010; 96: 1617 -21.

⁵⁶ Gospodarowicz M, Henson D, Hutter R, O’Sullivan B, Sobin L, Wittekind C. Prognostic factors in cancer. 2nd ed. John Wiley & Sons, 2001.

⁵⁷ Guyatt G, Rennie D, Meade M, Cook D. Users’ guides to the medical literature: a manual for evidence-based clinical practice. 2nd ed. AMA Press, 2008.

⁵⁸ Laupacis A, Wells G, Richardson WS, Tugwell P. Users’ guides to the medical literature. V. How to use an article about prognosis. Evidence-Based Medicine Working Group. JAMA 1994; 272: 234 -7.

⁵⁹ National Cancer Institute. Surveillance Epidemiology and End Results. http://seer.cancer.gov/ (updated 2012).

⁶⁰ Hemingway H, Philipson P, Chen R, Fitzpatrick NK, Damant J, Shipley M, et al. Evaluating the quality of research into a single prognostic biomarker: a systematic review and meta-analysis of 83 studies of C-Reactive protein in stable coronary artery disease. PLoS Med 2010; 7: e1000286.

⁶¹ Fox K, Garcia MA, Ardissino D, Buszman P, Camici PG, Crea F, et al. Guidelines on the management of stable angina pectoris: executive summary: the Task Force on the Management of Stable Angina Pectoris of the European Society of Cardiology. Eur Heart J 2006; 27: 1341 -81.

⁶² Repping-Wuts H, Uitterhoeve R, van RP, van AT. Fatigue as experienced by patients with rheumatoid arthritis (RA): a qualitative study. Int J Nurs Stud 2008;45:995-1002.