Content area
Full text
Introduction
Prediction models like those presented in the first article of this series, 1 use multiple predictors (covariates) to estimate the absolute probability or risk that a certain outcome is present (diagnostic prediction model) or will occur within a specific time period (prognostic prediction model) in an individual. 2-6 Estimated risks yielded by prediction models enable the stratification of individuals or groups of individuals by these risks. 7 Prediction models are usually developed to guide healthcare professionals in their decision-making about further management-including additional testing, initiating or withholding treatment(s)-and to inform individuals about their risks of having (diagnosis) or developing (prognosis) a particular disease or outcome. 8
Prediction modelling research as we recently described, 7-10 may distinguish three major phases including: (1) developing and internally validating a prediction model; (2) testing in, and if necessary, adjusting or updating the model for other individuals (external validation); (3) assessing the model's impact on therapeutic management and patient outcomes. The abundant publications on the development of prediction models were covered in the first article of this series. 1 Conversely, a relatively small number of studies have been published on the validation of prediction models and there are scarcely any showing whether implementing a prediction model has impact on healthcare providers' and individuals' behaviour or care, let alone on patient health outcomes or cost-effectiveness of care. 4 To show that a prediction model successfully predicts the outcome of interest in the development sample even when complemented with internal validation techniques, is not sufficient to confirm that a model is valuable. 7-10 Indeed, when applied to new individuals, the performance of prediction models is generally lower than the performance observed in the population from which the model was developed. Therefore, performance of developed and internally validated prediction models should still be tested or validated in new individuals before they are implemented in guidelines or applied in practice. 10
When a validation study shows disappointing results, researchers often reject the original prediction model and develop a new one from their own data. 11 12 However, the redeveloped model also often has several limitations, and multiple models for the same outcome create an impracticable situation where the user has to decide which model to use. For example, there are over 100 published...