Added value of clinical prediction rules for bacteremia in hemodialysis patients: An external validation study

Abstract

The most frequent cause of bacteremia in the general population is urinary tract infections caused by Escherichia coli [19–22], whereas the most frequent cause in HD is Staphylococcus aureus [23–25]. Since many blood test findings in HD patients are often affected by dialysis, the accuracy of items included in existing CPRs developed in the general population, such as blood counts and serum creatinine levels, may be greatly affected. The study was conducted in accordance with the ethical standards of the Declaration of Helsinki. Since all patient information analyzed in this study was retrospective, participants’ written informed consent was not required by the ethics committee. [...]an external consensus panel of infectious disease physicians with > 10 y clinical experience and Japanese Board of Infectious Disease certification who were blinded to the present study design determined whether a culture was contaminated or not based on the above definitions and their clinical expertise. Statistical analysis Validation cohorts.

About the Authors:

Sho Sasaki

Contributed equally to this work with: Sho Sasaki, Yoshihiko Raita

Roles Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Department of Nephrology, Iizuka Hospital, Fukuoka, Japan, Clinical Research Support Office, Iizuka Hospital, Fukuoka, Japan, Department of Healthcare Epidemiology, Kyoto University Graduate School of Public Health, Kyoto, Japan

ORCID logo https://orcid.org/0000-0002-6744-5931

Yoshihiko Raita

Contributed equally to this work with: Sho Sasaki, Yoshihiko Raita

Roles Data curation, Methodology, Supervision, Writing – original draft, Writing – review & editing

Affiliations Department of Nephrology, Okinawa Prefectural Chubu Hospital, Naha, Japan, Department of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America

Minoru Murakami

Roles Data curation, Supervision

Affiliation: Department of Nephrology, Saku Central Hospital, Nagano, Japan

ORCID logo https://orcid.org/0000-0003-0435-7106

Shungo Yamamoto

Roles Data curation, Methodology, Writing – original draft

Affiliations Department of Healthcare Epidemiology, Kyoto University Graduate School of Public Health, Kyoto, Japan, Department of Infectious Disease, Kyoto City Hospital, Kyoto, Japan

Kentaro Tochitani

Roles Data curation, Methodology, Writing – original draft

Affiliations Department of Healthcare Epidemiology, Kyoto University Graduate School of Public Health, Kyoto, Japan, Department of Infectious Disease, Kyoto City Hospital, Kyoto, Japan

Takeshi Hasegawa

Roles Methodology, Supervision, Writing – review & editing

Affiliations Office for Promoting Medical Research, Showa University, Tokyo, Japan, Division of Nephrology, Department of Medicine, Showa University Fujigaoka Hospital, Yokohama, Japan, Fukushima Medical University, Fukushima, Japan

ORCID logo https://orcid.org/0000-0003-0420-5583

Kiichiro Fujisaki

Roles Supervision, Writing – review & editing

Affiliation: Department of Nephrology, Iizuka Hospital, Fukuoka, Japan

Shunichi Fukuhara

Roles Supervision, Writing – review & editing

Affiliations Fukushima Medical University, Fukushima, Japan, Section of Clinical Epidemiology, Department of Community Medicine, Kyoto University, Kyoto, Japan, Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States of America

Introduction

Bacteremia is a disease with a high mortality rate [1–4]. Early diagnosis and treatment are keys to improving prognosis. However, due to the variety of clinical presentations of bacteremia, it is not always first on the list of possible diagnoses. In this context, several clinical predictive models (CPRs) of bacteremia in the general population have been developed [5–8]. Among these models, those developed by Shapiro et al. (Shapiro’s model) [9] have been widely validated and internationally recognized [10–12].

Patients on hemodialysis (HD) are known to have a higher morbidity and mortality rate from bacteremia compared to the general population. Previous cohort studies have shown that the incidence of bacteremia in patients on maintenance HD is 10.40–18.98 per 100 person-years [13–16], which is much higher than the incidence in the general population of 0.22 per 100 person-years [1]. The annual mortality due to sepsis, a severe complication of bacteremia, in HD patients is 50–100 times higher than that of the general population [17, 18].

The most frequent cause of bacteremia in the general population is urinary tract infections caused by Escherichia coli [19–22], whereas the most frequent cause in HD is Staphylococcus aureus [23–25]. Since many blood test findings in HD patients are often affected by dialysis, the accuracy of items included in existing CPRs developed in the general population, such as blood counts and serum creatinine levels, may be greatly affected.

[12]. In order to address the peculiarities of bacteremia among patients with HD, we developed a CPR specific to bacteremia in HD patients, the BAC-HD score [26].

Since the external validities of Shapiro’s model [12] and the BAC-HD score in patients on maintenance HD have not been verified, we assessed the diagnostic accuracy of these two prediction models for bacteremia.

Materials and methods

The present study was approved by the ethics committees of Iizuka Hospital (17167–436), Okinawa Prefectural Chubu Hospital (H28-51) and Saku General Hospital (R201701-01). The study was conducted in accordance with the ethical standards of the Declaration of Helsinki. Since all patient information analyzed in this study was retrospective, participants’ written informed consent was not required by the ethics committee. All data were fully anonymized before authors accessed them. We accessed the medical records to obtain data at Okinawa Prefectural Chubu Hospital from February 9th to February 13th, 2017, and at Saku General Hospital from February 24th to 26th, 2017. The study results are reported in accordance with the Standards for Reporting Diagnostic Accuracy (STARD) statement [27].

Study design and participants

We conducted a cross-sectional study of maintenance HD patients at two tertiary-care teaching hospitals.

Data were collected from medical records from January 2013 to December 2015 in each facility. We enrolled consecutive participants on maintenance HD who were aged ≥ 18 y with two sets of blood cultures drawn at admission because of suspicion of bacteremia. Exclusion criteria were participants who met any of the following items: 1) inpatients transferred from another hospital, 2) patients with a vintage of dialysis < 2 months, 3) patients also receiving peritoneal dialysis, and 4) patients receiving HD less than once a week.

Index tests

Two clinical prediction rules (CPR) for bacteremia were adopted as index tests.

The first CPR, the BAC-HD score, consists of the following 5 items: 1) body temperature ≥ 38.3°C, 2) heart rate ≥ 125, 3) C-reactive protein ≥ 10 × 104μg/L, 4) alkaline phosphatase > 6 μkat IU/L, and 5) no prior antibiotic use within the past week. Each item is regarded as 1 point, for a maximum total of 5 points.

Shapiro’s score is composed of “major criteria” defined as: 1) temperature > 39.5°C (3 points), 2) indwelling vascular catheter (2 points) or clinical suspicion of endocarditis (3 points); plus “minor criteria” (1 point each) defined as: 1) temperature 38.3–39.4°C (101–102.9°F), 2) age > 65 y, 3) chills, 4) vomiting, 5) hypotension (systolic blood pressure < 90 mmHg), 6) white blood cell count > 18 × 109/L, 7) bands > 5%, 8) platelets < 150 × 109/L, and 9) creatinine > 176.8 μmol/L.

Reference standard

The reference standard was bacteremia, as per the results of the admission blood cultures. Contamination was defined as: one of the two sets of culture bottles was positive, or cases where certain species of bacteria known to be contaminants, such as diphtheroids, Bacillus spp., Propionibacterium spp., Micrococci, Corynebacterium spp., and coagulase-negative Staphylococci were detected. Finally, an external consensus panel of infectious disease physicians with > 10 y clinical experience and Japanese Board of Infectious Disease certification who were blinded to the present study design determined whether a culture was contaminated or not based on the above definitions and their clinical expertise.

Statistical analysis

Validation cohorts.

Since there are no standard criteria for obtaining blood cultures, it was suspected that the selection of subjects may have differed depending on the facility. Therefore, two validation cohorts were set based on the logic that it is desirable to verify the validity at multiple facilities. Validation cohort 1 (360 subjects) and validation cohort 2 (96 subjects) were defined as patients at Okinawa Prefectural Chubu Hospital and Saku General Hospital, respectively.

Descriptive statistics.

We analyzed each item with the two CPRs and used proven bacteremia as a reference standard as well as other clinical information, including sex, blood pressure, respiratory rate, hemodialysis vintage, and presence of diabetes mellitus. Continuous and categorical variables are presented as the median (interquartile range: IQR) and number (percentage), respectively (Table 1).

[Figure omitted. See PDF.]

Table 1. Baseline characteristics.

https://doi.org/10.1371/journal.pone.0247624.t001

Basic model.

The basic model to assess the value of reclassification of CPRs was conducted using a logistic regression model with explanatory variables of sex (0: female, 1: male), age (y), mean arterial pressure (mmHg), heart rate, body temperature (°C), presence of diabetes mellitus, HD vintage (months), and white blood cell count (× 109/L). These items were selected by clinicians as those typically used when evaluating patients for bacteremia in their daily medical practice. In validation cohort 2, it was clear at the planning stage that the respiratory rate was often missing, so the respiratory rate was excluded from the basic model.

Added value of CPRs.

The discriminatory abilities of the basic model (Model 1), the model that added BAC-HD score (Model 2), or the model that added Shapiro’s score (Model 3) to the basic model were assessed by calculating the area under the receiver operating characteristic curve (AUC) using a logistic regression model. Calibration of each model was performed based on the slope and intercept of the calibration plot [28, 29].

The number of patients correctly reclassified by adding the CPR to the basic model is shown using Net Reclassification Improvement (NRI) (Table 2). The prediction probabilities in the three models were stratified based on the tertile of the prediction probabilities of the basic model, as low (< 0.08), intermediate (0.08–0.2), or high (> 0.2).

[Figure omitted. See PDF.]

Table 2. Reclassification table comparing the probability of bacteremia predicted by Model 2 (basic model + BAC-HD score) and Model 3 (basic model + Shapiro’s score).

https://doi.org/10.1371/journal.pone.0247624.t002

Assessment of performance.

To evaluate potential cut-off scores, we computed the sensitivity, specificity, likelihood ratio, positive predictive value, and negative predictive value for the CPRs. For brevity, only the values in validation cohort 1 are summarized.

Handling of missing values.

All missing values were addressed using multiple imputations by chained equation treated as missing at random; ten imputed datasets were created. Three logistic regression models were conducted on each of 10 datasets and combined with Rubin’s rule.

All statistical analyses were performed using Stata version 15.0 (Stata Corp., College Station, TX, USA).

Results

Study participants

As the final analytic cohort, there were 360 participants and 96 participants in validation cohort 1 and validation cohort 2, respectively, as shown in Fig 1.

[Figure omitted. See PDF.]

Fig 1. Study flow.

https://doi.org/10.1371/journal.pone.0247624.g001

Added diagnostic value of BAC-HD score and Shapiro’s score

In validation cohort 1, compared to the AUC of 0.69 (95% confidence interval [95% CI]: 0.59−0.80) of Model 1 (Basic model), the AUCs of Model 2 (Basic model + BAC-HD score) and Model 3 (Basic model + Shapiro’s score) increased to 0.8 (95% CI: 0.71−0.88) and 0.73 (95% CI: 0.63−0.83), respectively, as shown in Fig 2. In validation cohort 2, compared to the AUC of 0.81 (95% confidence interval [95% CI]: 0.68−0.94) of Model 1 (Basic model), the AUCs of Model 2 (Basic model + BAC-HD score) and Model 3 (Basic model + Shapiro’s score) increased to 0.83 (95% CI: 0.72–0.95) and 0.85 (95% CI: 0.76−0.94), respectively, as shown in Fig 2.

[Figure omitted. See PDF.]

Fig 2. Receiver operating characteristic curve.

A: results for validation cohort 1, B: results for validation cohort 2. Model 1 = basic model, Model 2 = basic model + BAC-HD score, Model 3 = basic model + Shapiro’s score, AUC: area under the receiver operating characteristic curve, 95% CI: 95% confidence interval.

https://doi.org/10.1371/journal.pone.0247624.g002

In validation cohort 1, the slopes (intercept) of the calibration plots of Models 1, 2, and 3 were 1.19 (−0.20), 1.09 (−0.01) and 1.15 (−0.01), respectively. In validation cohort 2, the slopes (intercept) of the calibration plots of Models 1, 2 and 3 were 1.03 (−0.01), 1.00 (−0.001) and 0.86 (0.03), respectively. Calibration plots are shown in Fig 3.

[Figure omitted. See PDF.]

Fig 3. Calibration plot.

Model 1 = basic model, Model 2 = basic model + BAC-HD score, Model 3 = basic model + Shapiro’s score.

https://doi.org/10.1371/journal.pone.0247624.g003

Reclassification by BAC-HD score and Shapiro’s score

In validation cohort 1 (N = 360), 138 of 142 patients predicted to have a low probability of bacteremia as a result of the basic model (Model 1) were not bacteremic (negative predictive value [NPV] = 97.1%). Thirteen of the 40 patients predicted to have a high probability of bacteremia had bacteremia (positive predictive value; PPV 32.5). Adding the BAC-HD score (Model 2) increased the number of patients predicted to be low-probability from 142 to 215, of whom 208 were not bacteremic (NPV 96.7%). The number of patients predicted to have a high probability of bacteremia was increased from 40 to 45, of whom 19 were bacteremic (PPV 42.2%). Adding Shapiro’s score (Model 3) increased the number of patients predicted to be low-probability from 142 to 177, of whom 169 were not bacteremic (NPV 95.4%). The number of patients predicted to be high probability was reduced from 40 to 36, of whom 14 were bacteremic (PPV 38.9%). The NRI values for addition of BAC-HD score and Shapiro’s score were 0.3 and 0.06, respectively, in validation cohort 1. The results for validation cohort 2 are shown in Table 2 and the NRIs for addition of BAC-HD score and Shapiro’s score were 0.27 and 0.13, respectively.

Assessment of performance

The sensitivities, specificities, likelihood ratios and predictive values for possible cut-off scores in CPRs in validation cohort 1 are shown in Table 3.

[Figure omitted. See PDF.]

Table 3. Assessment of performance of BAC-HD and Shapiro’s score in validation cohort 1.

https://doi.org/10.1371/journal.pone.0247624.t003

Discussion

This is the first study to validate and compare the external validity of the prediction rules for bacteremia in HD patients. BAC-HD showed an excellent added value, suggesting that it can be a useful tool for improving the diagnostic ability of bacteremia in dialysis patients. In addition, BAC-HD is a simple CPR consisting of five items, and has the advantage of being highly versatile in clinical settings.

On the other hand, although Shapiro’s score has many components, its added value was inferior to the BAC-HD score. The reason for this was considered to be the effects of three characteristics of patients on maintenance HD. The first is the difference in the bacteremia etiology. Bacteremia in the general population is often due to gram-negative rods (GNR) [19–22], usually due to urinary tract infections (UTI) [1, 30], while patients on HD often have gram-positive cocci (GPC) bacteremia due to cutaneous infections [23–25]. In our experience, UTI and BSI often have different clinical presentations. Therefore, it is considered that Shapiro’s score, a CPR developed in a population with a high rate of UTI, could not predict bacteremia in maintenance HD patients with a high rate of cutaneous infections.

This is consistent with our previous studies showing that the systemic inflammatory response syndrome (SIRS) criteria [31] and the quick Sequential (Sepsis-Related) Organ Failure Assessment (qSOFA) score [32, 33] were not useful in predicting bacteremia in HD patients. Second, the clinical information of patients on maintenance HD such as body weight, vital signs [34], electrolytes, blood urea nitrogen, and creatinine vary greatly between dialysis and non-dialysis days. Since bacteremia is assumed to develop regardless of the timing of dialysis, the predictive ability of these values may be impaired. Third, patients with end-stage renal failure are often immunocompromised and may have different clinical presentations [35].

This study has two strengths. First, since we verified the added value in two validation cohorts, the robustness of the results is likely increased. Second, the basic model was used to show the added value of CPRs. Although some external validation studies only showed discrimination and calibration of the CPR itself, this is not enough to evaluate the degree of improvement in predictive ability [36].

This study also has some limitations. First, since only Japanese patients with HD were included, its validity in other ethnic groups is unknown. On the other hand, AV-fistula (AVF) is used as vascular access in more than 93% [37] of Japanese on HD, satisfying the 65% goal of the fistula first initiative [38]. Since AVF is associated with a lower risk of infection compared to other vascular access methods, especially central venous catheters [39], it seems significant that added value was shown in a population with a high proportion of AVFs, which is the desired result. Second, since this study analyzed retrospective medical data, there are risks of bias caused by missing values or lower measurement accuracy. However, we performed multiple imputations for missing data to minimize such a bias [40]. Prospective studies are needed for better verification. Third, since bands, which were one of the components of Shapiro’s score, were not evaluated, the score may have been underestimated. Since a number of facilities cannot always measure the proportion of bands, such as facilities included in a previous validation study of Shapiro’s score [12], this is considered acceptable in terms of versatility. Since there are studies (including this present study) that have modified the items included in Shapiro’s score, future validation of these is awaited [41]. Fourth, since there were no standard criteria for when to obtain blood cultures, the possibility that there was some degree of arbitrariness in the decision to draw blood cultures cannot be denied. However, since we used data before the development of the BAC-HD score, it is unlikely that the items in the BAC-HD score influenced the decision on whether or not to draw blood cultures. Furthermore, it is possible that some of the patients whose blood cultures were not drawn included cases of bacteremia. Fifth, it is unclear whether blood cultures were collected from a central venous catheter (CVC) that was the site of vascular access. However, since it is unlikely that both sets of blood cultures were collected from the CVC, and the number of CVCs was small, we believe that the effect of this was not significant.

Conclusions

We verified the added value of the BAC-HD score and Shapiro’s score to the usual criteria for predicting bacteremia in HD patients. We suggest that either the BAC-HD score or Shapiro’s score may improve the accuracy of predicting bacteremia in patients on HD. Reclassification was better with the BAC-HD score.

Improving the diagnostic ability is expected to contribute to the early initiation of appropriate treatment and improve the prognosis of bacteremia.

Supporting information

S1 Table. Pathogens involved in bacteremia in validation cohorts 1 and 2.

https://doi.org/10.1371/journal.pone.0247624.s001

(DOCX)

S1 Data. Validation cohort 1.

https://doi.org/10.1371/journal.pone.0247624.s002

(XLS)

S2 Data. Validation cohort 2.

https://doi.org/10.1371/journal.pone.0247624.s003

(XLS)

Acknowledgments

The authors thank the JOINT-KD collaborators, Ryo Nishioka and Yasunori Suzuki, Department of Rheumatology, Kanazawa University Hospital, Kanazawa and Naomi Ako, Division of Nephrology and Hypertension, Department of Internal Medicine, St. Marianna University School of Medicine, Kawasaki for their intellectual support in the management of this study. We also thank Libby Cone, MD, MA, of DMC Corp. (www.dmed.co.jp) for editing drafts of this manuscript.

Citation: Sasaki S, Raita Y, Murakami M, Yamamoto S, Tochitani K, Hasegawa T, et al. (2021) Added value of clinical prediction rules for bacteremia in hemodialysis patients: An external validation study. PLoS ONE 16(2): e0247624. https://doi.org/10.1371/journal.pone.0247624

References

1. Nielsen SL, Pedersen C, Jensen TG, et al.; Decreasing incidence rates of bacteremia: a 9-year population-based study. J Infect 2014; 69(1):51–9. pmid:24576825

2. Ferrer R, Martin-Loeches I, Phillips G, et al.; Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program. Critical care medicine 2014; 42(8):1749–1755. pmid:24717459

3. Dellinger RP, Levy MM, Rhodes A, et al.; Surviving Sepsis Campaign: International Guidelines for Management of Severe Sepsis and Septic Shock: 2012. Critical Care Medicine 2013; 41(2):580–637. pmid:23353941

4. Levy MM, Artigas A, Phillips GS, et al.; Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. The Lancet infectious diseases 2012; 12(12):919–924. pmid:23103175

5. Ratzinger F, Dedeyan M, Rammerstorfer M, et al.; A risk prediction model for screening bacteremic patients: a cross sectional study. PLoS One 2014; 9(9):e106765. pmid:25184209

6. van Werkhoven CH, Huijts SM, Postma DF, et al.; Predictors of Bacteraemia in Patients with Suspected Community-Acquired Pneumonia. PLoS One 2015; 10(11):e0143817. pmid:26599636

7. Eliakim-Raz N, Bates DW, Leibovici L; Predicting bacteraemia in validated models—a systematic review. Clin Microbiol Infect 2015; 21(4):295–301. pmid:25677625

8. Takeshima T, Yamamoto Y, Noguchi Y, et al.; Identifying Patients with Bacteremia in Community-Hospital Emergency Rooms: A Retrospective Cohort Study. PLoS One 2016; 11(3):e0148078. pmid:27023336

9. Shapiro NI, Wolfe RE, Wright SB, et al.; Who needs a blood culture? A prospectively derived and validated prediction rule. J Emerg Med 2008; 35(3):255–64. pmid:18486413

10. Laukemann S, Kasper N, Kulkarni P, et al.; Can We Reduce Negative Blood Cultures With Clinical Scores and Blood Markers? Results From an Observational Cohort Study. Medicine (Baltimore) 2015; 94(49):e2264. pmid:26656373

11. Jessen MK, Mackenhauer J, Hvass AM, et al.; Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule. Eur J Emerg Med 2016; 23(1):44–9. pmid:25222426

12. Hodgson LE, Dragolea N, Venn R, et al.; An external validation study of a clinical prediction rule for medical patients with suspected bacteraemia. Emerg Med J 2016; 33(2):124–9. pmid:26246024

13. Hoen B, Paul-Dauphin A, Hestin D, et al.; EPIBACDIAL: a multicenter prospective study of risk factors for bacteremia in chronic hemodialysis patients. J Am Soc Nephrol 1998; 9(5):869–76. pmid:9596085

14. Dopirak M, Hill C, Oleksiw M, et al.; Surveillance of hemodialysis-associated primary bloodstream infections: the experience of ten hospital-based centers. Infect Control Hosp Epidemiol 2002; 23(12):721–4. pmid:12517013

15. Ishani A, Collins AJ, Herzog CA, et al.; Septicemia, access and cardiovascular disease in dialysis patients: the USRDS Wave 2 study. Kidney Int 2005; 68(1):311–8. pmid:15954922

16. Fysaraki M, Samonis G, Valachis A, et al.; Incidence, clinical, microbiological features and outcome of bloodstream infections in patients undergoing hemodialysis. Int J Med Sci 2013; 10(12):1632–8. pmid:24151435

17. Foley RN, Guo H, Snyder JJ, et al.; Septicemia in the United States dialysis population, 1991 to 1999. J Am Soc Nephrol 2004; 15(4):1038–45. pmid:15034107

18. Sarnak MJ, Jaber BL; Mortality caused by sepsis in patients with end-stage renal disease compared with the general population. Kidney Int 2000; 58(4):1758–64. pmid:11012910

19. Fontanarosa PB, Kaeberlein FJ, Gerson LW, et al.; Difficulty in predicting bacteremia in elderly emergency patients. Ann Emerg Med 1992; 21(7):842–8. pmid:1610043

20. Meyers BR, Sherman E, Mendelson MH, et al.; Bloodstream infections in the elderly. Am J Med 1989; 86(4):379–84. pmid:2929625

21. Elhanan G, Raz R, Pitlik SD, et al.; Bacteraemia in a community and a university hospital. J Antimicrob Chemother 1995; 36(4):681–95. pmid:8591943

22. Khayr WF, CarMichael MJ, Dubanowich CS, et al.; Epidemiology of bacteremia in the geriatric population. Am J Ther 2003; 10(2):127–31. pmid:12629591

23. Nielsen LH, Jensen-Fangel S, Benfield T, et al.; Risk and prognosis of Staphylococcus aureus bacteremia among individuals with and without end-stage renal disease: a Danish, population-based cohort study. BMC Infect Dis 2015; 15:6. pmid:25566857

24. Girndt M; Bacteraemia in haemodialysis patients—not always Staphylococcus aureus. Nephrol Dial Transplant 2015; 30(7):1055–7. pmid:26056175

25. Vandecasteele SJ, Boelaert JR, De Vriese AS; Staphylococcus aureus infections in hemodialysis: what a nephrologist should know. Clin J Am Soc Nephrol 2009; 4(8):1388–400. pmid:19590063

26. Sasaki S, Hasegawa T, Kawarazaki H, et al.; Development and Validation of a Clinical Prediction Rule for Bacteremia among Maintenance Hemodialysis Patients in Outpatient Settings. PLoS One 2017; 12(1):e0169975. pmid:28081211

27. Bossuyt PM, Reitsma JB, Bruns DE, et al.; STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Bmj 2015; 351:h5527. pmid:26511519

28. Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. John Wiley & Sons, Hoboken, NJ: 2013;chapter 5: 144–167

29. Steyerberg EW, Vickers AJ, Cook NR, et al.; Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21(1):128–38. pmid:20010215

30. Arpi M, Renneberg J, Andersen HK, et al.; Bacteremia at a Danish university hospital during a twenty-five-year period (1968–1992). Scand J Infect Dis 1995; 27(3):245–51. pmid:8539549

31. Uchida D, Sasaki S, Kawarazaki H, et al.; Systemic Inflammatory Response Syndrome Is Not an Indicator of Bacteremia in Hemodialysis Patients With Native Accesses: A Multicenter Study. Asaio j 2017; 63(4):501–506. pmid:27984318

32. Freund Y, Lemachatti N, Krastinova E, et al.; Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. Jama 2017; 317(3):301–308. pmid:28114554

33. Nishiwaki H, Sasaki S, Hasegawa T, et al.; External validation of the quick Sequential Organ Failure Assessment score for mortality and bacteraemia risk evaluation in Japanese patients undergoing haemodialysis: a retrospective multicentre cohort study. BMJ Open 2019; 9(7):e028856. pmid:31300504

34. Oosugi K, Fujimoto N, Dohi K, et al.; Hemodynamic and pathophysiological characteristics of intradialytic blood pressure elevation in patients with end-stage renal disease. Hypertens Res 2014; 37(2):158–65. pmid:24048483

35. Hauser AB, Stinghen AE, Kato S, et al.; Characteristics and causes of immune dysfunction related to uremia and dialysis. Perit Dial Int 2008; 28 Suppl 3:S183–7. pmid:18552253

36. Moons KG, de Groot JA, Linnet K, et al.; Quantifying the added value of a diagnostic test or marker. Clin Chem 2012; 58(10):1408–17. pmid:22952348

37. Pisoni RL, Zepel L, Fluck R, et al.; International Differences in the Location and Use of Arteriovenous Accesses Created for Hemodialysis: Results From the Dialysis Outcomes and Practice Patterns Study (DOPPS). American Journal of Kidney Diseases 2018; 71(4):469–478. pmid:29198387

38. Gold JA, Hoffman K; Fistula First: the National Vascular Access Improvement Initiative. Wmj 2006; 105(3):71–3. pmid:16749331

39. Ravani P, Gillespie BW, Quinn RR, et al.; Temporal risk profile for infectious and noninfectious complications of hemodialysis access. J Am Soc Nephrol 2013; 24(10):1668–77. pmid:23847278

40. van der Heijden GJ, Donders AR, Stijnen T, et al.; Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example. J Clin Epidemiol 2006; 59(10):1102–9. pmid:16980151

41. Wyss G, Berger S, Haubitz S, et al.; The Shapiro–Procalcitonin algorithm (SPA) as a decision tool for blood culture sampling: validation in a prospective cohort study. Infection 2020; 48: 523–533 pmid:32291611