Recalibration and Validation of the Charlson Comorbidity Index in Korean Incident Hemodialysis Patients

Abstract

Background

Weights assigned to comorbidities to predict mortality may vary based on the type of index disease and advances in the management of comorbidities. We aimed to develop a modified Charlson comorbidity index (CCI) in incident hemodialysis patients (mCCI-IHD), thereby improving risk stratification for mortality.

Methods

Data on 24,738 Koreans who received their first hemodialysis treatment between 2005 and 2008 were obtained from the Korean Health Insurance dataset. The mCCI-IHD score were calculated by summing up the weights which were assigned to individual comorbidities according to their relative prognostic significance determined by multivariate Cox proportional hazards model. The modified index was validated in an independent nationwide prospective cohort (n=1,100).

Results

The Cox proportional hazards model revealed that all comorbidities in the CCI except ulcers significantly predicted mortality. Thus, the mCCI-IHD included 14 comorbidities with re-assigned severity weights. In the validation cohort, both the CCI and the mCCI-IHD were correlated with mortality. However, the mCCI-IHD showed modest but significant increases in c statistics compared with the CCI at 6 months and 1 year. The analyses using continuous net reclassification improvement revealed that the mCCI-IHD improved net mortality risk reclassification by 24.6% (95% CI, 2.5-46.7; P=0.03), 26.2% (95% CI, 1.0-51.4; P=0.04) and 42.8% (95% CI, 4.9-80.8; P=0.03) with respect to the CCI at 6 months and 1 and 2 years, respectively.

Conclusions

The mCCI-IHD facilitates better risk stratification for mortality in incident hemodialysis patients compared with the CCI, suggesting that it may be a preferred index for use in clinical practice and the statistical analysis of epidemiological studies.

About the Authors:

Jae Yoon Park

Contributed equally to this work with: Jae Yoon Park, Myoung-Hee Kim

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Myoung-Hee Kim

Contributed equally to this work with: Jae Yoon Park, Myoung-Hee Kim

Affiliation: Department of Dental Hygiene, College of Health Science, Eulji University, Gyeonggi-do, Korea

Seung Seok Han

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Hyunjeong Cho

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Ho Kim

Affiliation: Department of Epidemiology and Biostatistics, School of Public Health, Seoul National University, Seoul, Korea

Dong-Ryeol Ryu

Affiliation: Department of Internal Medicine and Ewha Medical Research Institute, School of Medicine, Ewha Womans University, Seoul, Korea

Hyunwook Kim

Affiliation: Department of Internal Medicine, Wonkwang University College of Medicine Sanbon Hospital, Gyeonggi-do, Korea

Hajeong Lee

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Jung Pyo Lee

Affiliation: Department of Internal Medicine, Seoul National University Boramae Medical Center, Seoul, Korea

Chun-Soo Lim

Affiliation: Department of Internal Medicine, Seoul National University Boramae Medical Center, Seoul, Korea

Kyoung Hoon Kim

Affiliation: Department of Public Health, Graduate School, Korea University, Seoul, Korea

Kwon Wook Joo

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Yon Su Kim

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Dong Ki Kim

* E-mail: [email protected]

Affiliation: Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Korea

Clinical Research Center for End Stage Renal Disease (CRC for ESRD) Investigators

¶Membership of the Clinical Research Center for End Stage Renal Disease (CRC for ESRD) Investigators is provided in the Acknowledgments.

Introduction

Although the prognosis of dialysis patients has improved significantly in recent years, mortality rates for these patients remain several times higher than those for individuals in the general population [1]. Comorbidities in dialysis patients, which are highly prevalent and characterized by simultaneous coexistence, are often associated with substantially high mortality rates and socio-economic burden [1–6]. Thus, the prognostic impacts of comorbidities may influence diverse decisions during clinical practice and should be adjusted as strong confounding factors in statistical analyses. Consequently, comorbidities should be carefully assessed and summarized by an index that is intuitive to physicians for clinical and research uses. More importantly, such an index should reflect current advances regarding the management and outcomes of comorbidities and should be updated accordingly.

Among the current indices, the Charlson comorbidity index (CCI) which was developed in 1987, is the most popular and simple. It has been successfully validated in diverse chronic disease populations, including dialysis patients [7–10]. Nevertheless, there are several reasons to recalibrate and subsequently validate the index in dialysis patients to obtain better predictions regarding their mortality. Because the assigned weights of comorbidities in the CCI were generated from a general medical inpatient population, some comorbid conditions might have different significance in predicting mortality among dialysis patients [11, 12]. In addition, the management of dialysis patients and their comorbidities has advanced considerably, and the contributions of comorbidities to mortality rates might have changed as well [13]. Although some previous studies proposed modification of the CCI in dialysis patients, these modified indices were developed based on relatively small numbers of dialysis patients and were not externally validated through appropriate performance analyses [14, 15].

In the present study, we aimed to modify the CCI by recalibrating the weights of comorbidities in the CCI using a Korean national population-based registry database, to validate the index in a separate nationwide prospective end-stage renal disease (ESRD) cohort, and to compare its performance with the CCI.

Materials and Methods

Data source

Development cohort.

Deidentified data on all Koreans aged ≥18 years who received hemodialysis (HD) as their first modality of renal replacement therapy between January 2005 and December 2008 were obtained from the Korean Health Insurance database (n = 28,032). Data on all Korean HD patients could be reviewed because Koreans are obligated to register with the national insurance system, which consists of health insurance and medical aid. Subjects who died (n = 2,751), changed their modality of renal replacement (n = 229), or underwent kidney transplantation (n = 314) within 3 months after starting HD were excluded. Therefore, a total of 24,738 patients were included in the analysis to develop a modified CCI for incident HD patients (mCCI-IHD).

Validation cohort.

All patients aged ≥18 years who started HD as their first modality of renal replacement therapy between August 2008 and December 2012 in 31 centers of the Clinical Research Center (CRC) for ESRD and remained on HD for more than 3 months were initially screened for the validation cohort (n = 1,165). The CRC for ESRD is a nationwide multicenter prospective cohort of patients with ESRD in Korea (NCT00931970). Patients who died within 3 months of the commencement of hemodialysis (n = 65) were excluded from the analyses. A total of 1,100 incident HD patients were included to externally validate the mCCI-IHD.

Data collection

Comorbidities were defined based on the International Classification of Diseases, 10th Revision (ICD-10) (S1 Table) [16]. For the development study, information on comorbidities that constitute the CCI, including myocardial infarction (MI), congestive heart failure (CHF), peripheral vascular disease (PVD), cerebrovascular disease, dementia, chronic pulmonary disease, connective tissue disease, ulcer disease, mild liver disease, diabetes, hemiplegia, diabetes with end-organ damage, any tumor (including leukemia and lymphoma), moderate to severe liver disease and metastatic solid tumors were derived by extracting ICD-10 codes leading up to the initiation of dialysis therapy [17]. The comorbidity data of the validation cohort were ascertained by clinicians of CRC for ESRD through history taking and review of electronic health record. Acquired immunodeficiency syndrome was not included because no patient was so diagnosed in either cohort. Mortality data were obtained from the Statistics Korea and National Health Insurance Claims databases. In the validation cohort, demographic characteristics included body mass index (BMI), health security system, primary cause of ESRD, and timing of referral to nephrology; early referral was defined by HD initiation occurring more than 12 months after the first encounter with a nephrologist. The laboratory data included pre-dialysis fasting blood samples taken 1 month after the initial HD and included the following: hemoglobin, albumin, calcium, and phosphorus.

Statistical analyses

All statistical analyses were performed using SAS version 9.3 (SAS Institute, Cary, NC) or R version 3.0.3 (R Development Core Team, Austria). The data were summarized according to means±SDs or medians±IQRs for continuous variables and proportions for categorical variables. The multivariate Cox proportional hazards model was used to develop new comorbidity weights. We computed adjusted HRs and β coefficients while accounting for age, sex, and all 15 comorbidities simultaneously. We excluded an unnecessary item of severe renal disease given its presence in all of the study subjects. Prognostic weights were derived by dividing each β coefficient by the lowest one with statistical significance and rounding to the nearest integer [18]. Following this procedure, a comorbidity score was calculated for each subject by summing up the weights. The prognostic powers of the CCI and mCCI-IHD were compared by estimating Kaplan-Meier curves and multivariate Cox regression models. To compare the capacity for discrimination between the indices, a c statistic was calculated using a receiver-operating characteristic curve [19]. Additionally, the continuous net reclassification improvement (cNRI) was calculated to determine the extent to which the mCCI-IHD improved mortality predictions compared to the CCI [20], age-adjusted CCI [21] and previously modified CCI (mCCI) in ESRD patients [15]. Logistic regression models were first adopted to calculate cNRI. In the fitted logistic models, the same covariates were entered by adding two scores separately using the function improveprob in R [22]. In all analyses, a P<0.05 was considered statistically significant.

Ethical aspects

The study protocol complied with the Declaration of Helsinki and received full approval from the institutional review board (IRB) at Seoul National University Hospital (H-1405-060-579). The study protocol of CRC for ESRD was approved by the IRB at each participating center [The Catholic University of Korea, Bucheon St. Mary’s Hospital; The Catholic University of Korea, Incheon St. Mary’s Hospital; The Catholic University of Korea, Seoul St. Mary’s Hospital; The Catholic University of Korea, St. Mary’s Hospital; The Catholic University of Korea, St. Vincent’s Hospital; The Catholic University of Korea, Uijeongbu St. Mary’s Hospital; Cheju Halla General Hospital; Chonbuk National University Hospital; Chonnam National University Hospital; Chung-Ang University Medical Center; Chungbuk National University Hospital; Chungnam National University Hospital; Dong-A University Medical Center; Ehwa Womens University Medical Center; Fatima Hospital, Daegu; Gachon University Gil Medical Center; Inje University Pusan Paik Hospital; Kyungpook National University Hospital; Kwandong University College of Medicine, Myongji Hospital; National Health Insurance Corporation Ilsan Hospital; National Medical Center; Pusan National University Hospital; Samsung Medical Center, Seoul; Seoul Metropolitan Government, Seoul National University, Boramae Medical Center; Seoul National University Hospital; Seoul National University, Bundang Hospital; Yeungnam University Medical Center; Yonsei University, Severance Hospital; Yonsei University, Gangnam Severance Hospital; Ulsan University Hospital; Wonju Christian Hospital (in alphabetical order)], and all patients provided their written informed consent.

Results

Development of the new comorbidity index (mCCI-IHD) for the prediction of mortality

Baseline characteristics of the development cohort are listed in Table 1. The mean age at initiation of HD therapy was 57.9 years and 59.5% of patients were men. The median follow-up duration was 47.7 months. A total of 76.3% of the subjects had one or more comorbidities. Among the 15 comorbidities, diabetes (with and without end-organ damage) was most prevalent (49.6%), followed by chronic pulmonary disease (16.3%), CHF (14.1%), and cerebrovascular disease (13.0%). The survival rates of the subjects were 92.4%, 87.5%, 78.9%, and 72.3% at 6 months and 1, 2, and 3 years of follow-up, respectively.

[Figure omitted. See PDF.]

Table 1. Baseline characteristics of the development and validation cohorts.

https://doi.org/10.1371/journal.pone.0127240.t001

Table 2 lists the adjusted HRs, β coefficients and weights for each comorbidity. All comorbidities except ulcers significantly predicted mortality. The relative weights of comorbidities were calculated by their ratios with the lowest β coefficient of chronic pulmonary disease. As expected, metastatic solid tumors were the strongest predictor of mortality among comorbidities; accordingly, the weight assigned to this condition was the highest. Moderate to severe liver disease, any tumor (including leukemia and lymphoma), diabetes with end-organ damage, dementia, and cerebrovascular disease followed consecutively.

[Figure omitted. See PDF.]

Table 2. Weights for comorbidities in the development cohort.

https://doi.org/10.1371/journal.pone.0127240.t002

The summed scores of the original and recalibrated weights were applied to each patient in the development cohort. In the Cox proportional hazards model adjusted for age and sex, the HRs of the CCI and mCCI-IHD scores for mortality were 1.42 (95% CI, 1.39–1.45; P<0.0001) and 1.72 (95% CI, 1.66–1.78; P<0.0001), respectively. Based on the CCI and mCCI-IHD scores, the subjects were further collapsed into 4 risk groups according to the following criteria: no comorbidity, ≤50th percentile, 50th-90th percentile, and >90th percentile. To determine the cutoff values of the comorbidity scores for each risk group, we examined the distributions of the scores using a histogram (Fig 1). The cutoff values of the CCI scores corresponding to the 50th and 90th percentiles were 4 and 6, whereas those of the mCCI-IHD scores corresponding to the same percentiles were 4 and 9. Fig 2 shows the Kaplan-Meier curves for the CCI and mCCI-IHD differentiated by the 4 risk groups in the development cohort. The appearance of survival curves across increasing comorbidity scores was similar for both indices, with anticipated poorer survival for higher scores (P<0.001).

[Figure omitted. See PDF.]

Fig 1. Distribution of the CCI and mCCI-IHD scores for the development cohort.

(A) Distribution of the CCI scores (n = 18,872), excluding patients with no comorbidity (n = 5,866). (B) Distribution of the mCCI-IHD scores (n = 18,087), excluding patients with no comorbidity (n = 6,651). The y-axis shows the number of subjects. The solid line represents a density curve, calculated by approximation, to identify the overall pattern and deviation. The vertical dotted lines (red) represents the 50th and 90th percentile values. CCI, Charlson comorbidity index; mCCI-IHD, modified Charlson comorbidity index for incident hemodialysis patients.

https://doi.org/10.1371/journal.pone.0127240.g001

[Figure omitted. See PDF.]

Fig 2. Survival curves obtained using the Kaplan-Meier method in the development cohort differentiated by 4 risk groups for CCI (A) and mCCI-IHD (B).

CCI, no comorbidity (n = 5,866, score 2); ≤50th percentile (n = 10,501, scores 3–4); 50th-90th percentile (n = 6,663, scores 5–6); and >90th percentile (n = 1,708, score ≥7). mCCI-IHD, no comorbidity (n = 6,651, score 0); ≤50th percentile (n = 9,740, scores 1–4); 50th-90th percentile (n = 7,072, scores 5–9); and >90th percentile (n = 1,275, score ≥10). CCI, Charlson comorbidity index; mCCI-IHD, modified Charlson comorbidity index for incident hemodialysis patients.

https://doi.org/10.1371/journal.pone.0127240.g002

Application and validation of the mCCI-IHD

Table 1 lists the characteristic of the validation cohort (n = 1,100). The majority of subjects were men, with a mean age of 60.7 years. A total of 143 mortalities (13.0%) were observed during the median follow-up of 22.0 months. Along with the development cohort, diabetes was the most common comorbidity (57.3%), followed by CHF (13.6%), cerebrovascular disease (13.1%), and PVD (10.4%). The mCCI-IHD and CCI scores were then calculated for patients in the validation cohort. Fig 3 shows the survival curves obtained using the Kaplan-Meier method and multivariate Cox regression analysis (adjusted for age, sex, health security system, timing of referral to nephrologists, primary cause of ESRD, BMI, hemoglobin, albumin, calcium, and phosphorus) in the validation cohort, which were differentiated by the 4 risk groups of the CCI and mCCI-IHD. The risk groups based on each index provided some discrimination for survival, indicating that increasing comorbidity index scores were associated with lower cumulative survival. However, multivariate-adjusted survival curves for the mCCI-IHD risk groups showed a highly significant linear gradient and were quite well separated, whereas the model using the CCI seemed less able to distinguish among the three lowest-risk groups (Fig 3).

[Figure omitted. See PDF.]

Fig 3. Survival curves obtained using the Kaplan-Meier method (A and B) and Cox regression analysis (C and D) in the validation cohort differentiated by the 4 risk groups of CCI (A and C) and mCCI-IHD (B and D).

CCI, score 2 (n = 236, 21.5%); scores 3–4 (n = 537, 48.8%); scores 5–6 (n = 234, 21.3%); and score ≥7 (n = 93, 8.5%). mCCI-IHD, score 0 (n = 247, 22.5%); scores 1–4 (n = 480, 43.6%); scores 5–9 (n = 290, 26.4%); and score ≥10 (n = 83, 7.5%). aAdjusted for age, sex, health security system, timing of referral to nephrologist, primary cause of end-stage renal disease, body mass index, hemoglobin, albumin, calcium, and phosphorus. CCI, Charlson comorbidity index; mCCI-IHD, modified Charlson comorbidity index for incident hemodialysis patients.

https://doi.org/10.1371/journal.pone.0127240.g003

As detailed in Table 3, increased c statistics and cNRI for the mCCI-IHD supported better performance characteristics of the mCCI-IHD compared to the CCI. The mCCI-IHD exhibited modest but significant increases in c statistics compared with the CCI at 6 months and 1 year but not at 2 years in multivariate analyses adjusted for age, sex, health security system, timing of referral to nephrologist, primary cause of ESRD, BMI, hemoglobin, albumin, calcium, and phosphorus. The analyses using cNRI revealed that the mCCI-IHD improved net mortality risk reclassification by 24.6% (95% CI, 2.5–46.7; P = 0.03), 26.2% (95% CI, 1.0–51.4; P = 0.04) and 42.8% (95% CI, 4.9–80.8; P = 0.03) relative to the CCI at 6 months and 1 and 2 years, respectively, after adjustment for the above-mentioned confounders. This finding indicates that the mCCI-IHD may provide better risk stratification for mortality in incident HD patients than the CCI. Additionally, the mCCI-IHD also improved the predictability for mortality compared with the age-adjusted CCI (cNRI, 25.6%; P = 0.04 and cNRI, 45.7%; P = 0.02 at 1 and 2 years, respectively) and the mCCI (cNRI, 48.0%; P = 0.02 at 2 years) after adjusting the confounding factors.

[Figure omitted. See PDF.]

Table 3. Model performance of the mCCI-IHD at various time points during the follow-up of the validation cohort.

https://doi.org/10.1371/journal.pone.0127240.t003

Discussion

In the present study, we modified the CCI using population-based registry data that included nearly all Korean patients who began HD between 2005 and 2008 to develop a comorbidity index that provides better risk stratification in incident HD patients. In addition, we validated the usefulness of the modified index in an independent prospective cohort by comparing its performance with the original index. To our knowledge, this is the first study to recalibrate the CCI using a large sample of incident HD patients and to validate the index using an independent nationwide prospective observational cohort of ESRD patients.

Despite highly prevalent comorbidities and their influence on mortality rates in ESRD patients, it has not yet been determined which comorbidities significantly affect mortality and what weight each comorbidity carries when predicting mortality. Therefore, many previous studies have examined the influence of comorbidity on survival in patients with ESRD [23–25], and several comorbidity instruments such as the CCI [14, 26], the index of co-existent disease (ICED) [27], the Davies index [14], and the Khan index [4, 14] have been validated in dialysis patients. Among the indices, the ICED has been shown to be a valid method of assessing comorbidity in dialysis patients and appears superior to the others based on its discriminatory ability [23, 28, 29]. However, this index consists of more than 100 variables, which makes it largely inapplicable for clinical practice or statistical analysis. The Davies index, which consists of 7 comorbidities, is a simple and clinically applicable instrument [30]. However, all the comorbidities in this index are assigned equal weights based on ill-defined criteria, which means there might be a lack of consideration for the differential impacts of the comorbidities on mortality. The Khan index also has limitations due to its reliance on weak evidence for selecting comorbidities as predictors of mortality and its use of equal weights for each comorbidity [4]. In contrast, the CCI is the most widely used and validated because of its convenience and relatively well-defined weights for comorbidities. Nevertheless, the CCI remains highly disputed as an index to be applied directly to current ESRD populations because the weights for certain comorbidities in the CCI may be overestimated when considering the contemporary management of comorbidities. Thus, the applicability of the CCI to current clinical practice or statistical analysis may be tenuous. Indeed, the adjusted HRs of comorbidities in the present study, especially in the case of cardiovascular comorbidities, which are the main causes of death in this population, were lower than those in the original study of Charlson et al. [26] (i.e., MI [1.21 versus 1.40], CHF [1.24 versus 1.30], PVD [1.10 versus 1.30], hemiplegia [1.29 versus 1.30], and cerebrovascular disease [1.34 versus 1.40]). Similar patterns were found when the results presented here were compared with HRs in a study from the early 2000s, which modified the CCI based on a relatively small number of ESRD patients (n = 237) [15]. Conversely, a recent study using data from the Dialysis Outcomes and Practice Patterns Study revealed HRs for cardiovascular comorbidities similar to those calculated in the present study, suggesting advances in the management of the comorbidities and subsequent improved outcomes since the 1980s [3]. While the HRs have decreased, the weights of the cardiovascular comorbidities were assigned higher values in the mCCI-IHD than in the CCI (i.e., MI [2 versus 1], CHF [2 versus 1], PVD [1 in both indices], hemiplegia [3 versus 2], and cerebrovascular disease [3 versus 1]). Given that cardiovascular comorbidities are the most common cause of death in this population [1, 31], this finding suggests that the mCCI-IHD properly reflects the differential significance of cardiovascular comorbidities in this population, thereby providing superior performance in mortality risk stratification.

The results of performance analyses demonstrated that the mCCI-IHD exhibited significant but modest improvements in c statistics at 6 months and 1 year compared with the CCI, exhibiting increases of only 0.03–0.04. In this regard, it has become increasingly recognized that increases in c statistics of a test model with a large effect size may be small when compared to a strong baseline model such as the CCI [32, 33]. Thus, even though it is a popular metric through which to capture discrimination, c statistics may not be useful for comparing the performance of two risk prediction models [34–36]. In contrast, cNRI has been proposed to assess the degree of improvement in discrimination of risk prediction between the models and is not strongly affected by the effect size of the baseline model [32, 33]. Furthermore, this metric can capture the degree of improvement in discrimination after accounting for correlation with other variables in the risk prediction model [32]. Therefore, we additionally calculated the cNRI to quantify the magnitude of correct reclassification, and the mCCI-IHD showed significant positive values compared with the CCI, age-adjusted CCI, and mCCI after adjustment for socio-economic, clinical and laboratory parameters related to the survival of HD patients [1, 37–40]. These findings indicate that the mCCI-IHD may provide better risk stratification for mortality in incident HD patients compared with the original and previously modified CCI.

Our study had limitations. First, our results from the development cohort were derived from the administrative database of the national insurance system. Although the administrative data have advantages, such as conservation of time and resources and consistency in diagnosis due to the uniform use of the ICD-10, there may be inappropriate diagnoses due to the drawbacks of the coding system and the physicians’ preference toward diagnostic codes with higher reimbursements [41]. In addition, this dataset did not contain parameters such as biochemical measures and socioeconomic status. Therefore, we could not adjust these variables in the analysis for recalibrating the weights of comorbidities. Second, because there was no patient with acquired immunodeficiency syndrome in either cohort, this condition was not taken into consideration. Third, we excluded those subjects who died within 3 months after starting HD to rule out patients with critical illness or terminal conditions with renal failure. However, this could bias the assigned severity weights of the comorbidities because of a higher mortality in early dialysis period. Fourth, the mCCI-IHD was derived from a single ethnic group of Korean HD patients. Therefore, generalizability of the index can be limited. Nonetheless, a major strength of the present study is that patient mortality was determined using a large nationwide, population-based data set. Furthermore, the modified index was validated using an independent database. However, additional validation using disease-specific cohorts and other national data may be necessary.

In conclusion, we propose a new scoring system that provides better risk stratification for mortality in incident HD patients compared with the CCI after adjusting for socio-economic, clinical and laboratory parameters. Therefore, it may be a preferred index for use in clinical practice and the statistical analysis of epidemiological studies.

Supporting Information

[Figure omitted. See PDF.]

S1 Table. ICD-10 coding algorithms for Charlson comorbidities.

https://doi.org/10.1371/journal.pone.0127240.s001

(DOCX)

Acknowledgments

This work was supported by a grant from the Korean Healthcare Technology R&D Project, Ministry for Health and Welfare, Republic of Korea (HI10C2020).

The members of the Clinical Research Center for End Stage Renal Disease (CRC for ESRD) are: Do J.Y. at Yeungnam University Medical Center; Song S.H. at Pusan National University Hospital; Kim S.E. at Dong-a University Medical Center; Kim S.H. at Daegu Fatima Hospital; Kim Y.H. at Inje University Paik Hospital; Lee J.S. at Ulsan University Hospital; Jin H.J. at Seoul National University Bundang Hospital; Lim C.S. and Lee J.P. at Seoul National University Boramae Medical Center; Chang J.H. at Gachon University Gil Medical Center; Yoo T.H., Park J.T. and Oh H.J. at Yonsei University Severance Hospital; Park H.C. at Yonsei University Gangnam Severance Hospital; Chang T.I. at National Health Insurance Corporation Ilsan Hospital; Ryu D.R. at Ehwa Womens University Mokdong Hospital; Oh D.J. at Kwandong University Myongji Hospital; Chang Y.S. at The Catholic University of Korea St. Mary's Hospital; Kim Y.O. at The Catholic University of Korea Uijeongbu St. Mary's Hospital; Kim S.H. at Chung-Ang University Medical Center; Jin D.C. at The Catholic University of Korea St. Vincent's Hospital; Kim Y.K. at The Catholic University of Korea, Bucheon St. Mary's Hospital; Kim H. Y. at Chungbuk National University Hospital; Kim W. at Chonbuk National University Hospital; Lee K.W. at Chungnam National University Hospital; and Lee C.S. at Cheju Halla General Hospital.

Author Contributions

Conceived and designed the experiments: DKK. Performed the experiments: MK Ho Kim. Analyzed the data: MK Ho Kim JYP DKK. Contributed reagents/materials/analysis tools: SSH DRR Hyunwook Kim HL JPL HC KHK CSL KWJ YSK. Wrote the paper: JYP DKK.

Citation: Park JY, Kim M-H, Han SS, Cho H, Kim H, Ryu D-R, et al. (2015) Recalibration and Validation of the Charlson Comorbidity Index in Korean Incident Hemodialysis Patients. PLoS ONE 10(5): e0127240. https://doi.org/10.1371/journal.pone.0127240

References

1. Collins AJ, Foley RN, Chavers B, Gilbertson D, Herzog C, Ishani A, et al. US Renal Data System 2013 Annual Data Report. Am J Kidney Dis. 2014;63(1 Suppl):A7. pmid:24360288.

2. Longenecker JC, Coresh J, Klag MJ, Levey AS, Martin AA, Fink NE, et al. Validation of comorbid conditions on the end-stage renal disease medical evidence report: the CHOICE study. Choices for Healthy Outcomes in Caring for ESRD. J Am Soc Nephrol 2000;11(3):520–9. pmid:10703676.

3. Bradbury BD, Fissell RB, Albert JM, Anthony MS, Critchlow CW, Pisoni RL, et al. Predictors of early mortality among incident US hemodialysis patients in the Dialysis Outcomes and Practice Patterns Study (DOPPS). Clin J Am Soc Nephrol. 2007;2(1):89–99. Epub 2007/08/21. pmid:17699392.

4. Khan IH, Catto GR, Edward N, Fleming LW, Henderson IS, MacLeod AM. Influence of coexisting disease on survival on renal-replacement therapy. Lancet. 1993;341(8842):415–8. Epub 1993/02/13. 0140-6736(93)93003-J [pii]. pmid:8094182.

5. Khan IH. Comorbidity: the major challenge for survival and quality of life in end-stage renal disease. Nephrol Dial Transplant. 1998;13 Suppl 1:76–9. Epub 1998/03/21. pmid:9507503.

6. Liu J, Huang Z, Gilbertson DT, Foley RN, Collins AJ. An improved comorbidity index for outcome analyses among dialysis patients. Kidney Int. 2010;77(2):141–51. pmid:19907414.

7. Radovanovic D, Seifert B, Urban P, Eberli FR, Rickli H, Bertel O, et al. Validity of Charlson Comorbidity Index in patients hospitalised with acute coronary syndrome. Insights from the nationwide AMIS Plus registry 2002–2012. Heart. 2014;100(4):288–94. pmid:24186563.

8. Frenkel WJ, Jongerius EJ, Mandjes-van Uitert MJ, van Munster BC, de Rooij SE. Validation of the Charlson Comorbidity Index in acutely hospitalized elderly adults: a prospective cohort study. J Am Geriatr Soc. 2014;62(2):342–6. pmid:24521366.

9. Reyes C, Estrada P, Nogues X, Orozco P, Cooper C, Diez-Perez A, et al. The impact of common co-morbidities (as measured using the Charlson index) on hip fracture risk in elderly men: a population-based cohort study. Osteoporosis Int. 2014;25(6):1751–8. pmid:24676845.

10. Wu PH, Lin YT, Lee TC, Lin MY, Kuo MC, Chiu YW, et al. Predicting mortality of incident dialysis patients in Taiwan—a longitudinal population-based study. PLoS One. 2013;8(4):e61930. Epub 2013/04/30. pmid:23626754; PubMed Central PMCID: PMC3633990.

11. Miskulin D, Bragg-Gresham J, Gillespie BW, Tentori F, Pisoni RL, Tighiouart H, et al. Key comorbid conditions that are predictive of survival among hemodialysis patients. Clin J Am Soc Nephrol. 2009;4(11):1818–26. pmid:19808231; PubMed Central PMCID: PMC2774950.

12. Hall SF. A user's guide to selecting a comorbidity index for clinical research. J Clin Epidemiol. 2006;59(8):849–55. Epub 2006/07/11. S0895-4356(05)00436-1 [pii] pmid:16828679.

13. Quan H, Li B, Couris CM, Fushimi K, Graham P, Hider P, et al. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173(6):676–82. pmid:21330339.

14. van Manen JG, Korevaar JC, Dekker FW, Boeschoten EW, Bossuyt PM, Krediet RT. How to adjust for comorbidity in survival studies in ESRD patients: a comparison of different indices. Am J Kidney Dis. 2002;40(1):82–9. Epub 2002/06/28. S0272-6386(02)00011-2 [pii] pmid:12087565.

15. Hemmelgarn BR, Manns BJ, Quan H, Ghali WA. Adapting the Charlson Comorbidity Index for use in patients with ESRD. Am J Kidney Dis. 2003;42(1):125–32. Epub 2003/06/28. doi: S0272638603004153 [pii]. pmid:12830464.

16. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–9. pmid:16224307.

17. Charlson M, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol. 1994;47(11):1245–51. Epub 1994/11/01. pmid:7722560.

18. Sullivan LM, Massaro JM, D'Agostino RB Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med. 2004;23(10):1631–60. pmid:15122742.

19. Harrell FE Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87. pmid:8668867.

20. Pencina MJ, D'Agostino RB Sr., D'Agostino RB Jr., Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72; discussion 207–12. pmid:17569110.

21. Beddhu S, Bruns FJ, Saul M, Seddon P, Zeidel ML. A simple comorbidity scale predicts clinical outcomes and costs in dialysis patients. Am J Med. 2000;108(8):609–13. Epub 2000/06/17. pmid:10856407.

22. Pencina MJ, D'Agostino RB Sr., Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. pmid:21204120; PubMed Central PMCID: PMC3341973.

23. Miskulin DC. Predicting 1 year mortality in an outpatient haemodialysis population: a comparison of comorbidity instruments. Nephrol Dial Transplant. 2004;19(2):413–20. pmid:14736967

24. Leavey SF, Strawderman RL, Jones CA, Port FK, Held PJ. Simple nutritional indicators as independent predictors of mortality in hemodialysis patients. Am J Kidney Dis. 1998;31(6):997–1006. Epub 1998/06/19. doi: S0272638698001516 [pii]. pmid:9631845.

25. Wallen MD, Radhakrishnan J, Appel G, Hodgson ME, Pablos-Mendez A. An analysis of cardiac mortality in patients with new-onset end-stage renal disease in New York State. Clin Nephrol. 2001;55(2):101–8. Epub 2001/03/28. pmid:11269672.

26. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. Epub 1987/01/01. pmid:3558716.

27. Athienites NV, Miskulin DC, Fernandez G, Bunnapradist S, Simon G, Landa M, et al. Comorbidity assessment in hemodialysis and peritoneal dialysis using the index of coexistent disease. Semin Dial. 2000;13(5):320–6. Epub 2000/10/03. pmid:11014695.

28. Miskulin DC, Athienites NV, Yan G, Martin AA, Ornt DB, Kusek JW, et al. Comorbidity assessment using the Index of Coexistent Diseases in a multicenter clinical trial. Kidney Int. 2001;60(4):1498–510. pmid:11576365.

29. Miskulin DC, Meyer KB, Athienites NV, Martin AA, Terrin N, Marsh JV, et al. Comorbidity and other factors associated with modality selection in incident dialysis patients: the CHOICE Study. Choices for Healthy Outcomes in Caring for End-Stage Renal Disease. Am J Kidney Dis. 2002;39(2):324–36. pmid:11840373.

30. Davies SJ, Russell L, Bryan J, Phillips L, Russell GI. Comorbidity, urea kinetics, and appetite in continuous ambulatory peritoneal dialysis patients: their interrelationship and prediction of survival. Am J Kidney Dis. 1995;26(2):353–61. Epub 1995/08/01. doi: 0272-6386(95)90657-6 [pii]. pmid:7645541.

31. Jin DC, Han JS: Renal replacement therapy in Korea. Kidney Res Clin Pract 33(1): 9–18, 2014

32. Pencina MJ, D'Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012;176(6):473–81. pmid:22875755; PubMed Central PMCID: PMC3530349.

33. Janes H, Pepe MS, Gu W. Assessing the value of risk predictions by using risk stratification tables. Ann Intern Med. 2008;149(10):751–60. pmid:19017593; PubMed Central PMCID: PMC3091826.

34. D'Agostino RB GJ, Schmidt CH, Terrin N. Measures for evaluating model performance. Proceedings of the Biometrics Section. 1997:253–8.

35. Ware JH. The limitations of risk factors as prognostic tools. N Engl J Med. 2006;355(25):2615–7. pmid:17182986.

36. Pencina MJ, D'Agostino RB Sr., Demler OV. Novel metrics for evaluating improvement in discrimination: net reclassification and integrated discrimination improvement for normal variables and nested models. Stat Med. 2012;31(2):101–13. pmid:22147389; PubMed Central PMCID: PMC3341978.

37. Selim G, Stojceva-Taneva O, Spasovski G, Tozija L, Grozdanovski R, Georgievska-Ismail L, et al. Timing of nephrology referral and initiation of dialysis as predictors for survival in hemodialysis patients: 5-year follow-up analysis. Int Urol Nephrol. 2014. pmid:25099522.

38. Kopple JD. Effect of nutrition on morbidity and mortality in maintenance dialysis patients. Am J Kidney Dis. 1994;24(6):1002–9. pmid:7985661.

39. Kopple JD, Zhu X, Lew NL, Lowrie EG. Body weight-for-height relationships predict mortality in maintenance hemodialysis patients. Kidney Int. 1999;56(3):1136–48. pmid:10469384.

40. Honda H, Qureshi AR, Heimburger O, Barany P, Wang K, Pecoits-Filho R, et al. Serum albumin, C-reactive protein, interleukin 6, and fetuin a as predictors of malnutrition, cardiovascular disease, and mortality in patients with ESRD. Am J Kidney Dis. 2006;47(1):139–48. pmid:16377395.

41. Miskulin D. Characterizing comorbidity in dialysis patients: principles of measurement and applications in risk adjustment and patient care. Perit Dial Int. 2005;25(4):320–32. pmid:16022085.