Automated Detection of External Ventricular and Lumbar Drain-Related Meningitis Using Laboratory and Microbiology Results and Medication Data

Abstract

Objective

Monitoring of healthcare-associated infection rates is important for infection control and hospital benchmarking. However, manual surveillance is time-consuming and susceptible to error. The aim was, therefore, to develop a prediction model to retrospectively detect drain-related meningitis (DRM), a frequently occurring nosocomial infection, using routinely collected data from a clinical data warehouse.

Methods

As part of the hospital infection control program, all patients receiving an external ventricular (EVD) or lumbar drain (ELD) (2004 to 2009; n = 742) had been evaluated for the development of DRM through chart review and standardized diagnostic criteria by infection control staff; this was the reference standard. Children, patients dying <24 hours after drain insertion or with <1 day follow-up and patients with infection at the time of insertion or multiple simultaneous drains were excluded. Logistic regression was used to develop a model predicting the occurrence of DRM. Missing data were imputed using multiple imputation. Bootstrapping was applied to increase generalizability.

Results

537 patients remained after application of exclusion criteria, of which 82 developed DRM (13.5/1000 days at risk). The automated model to detect DRM included the number of drains placed, drain type, blood leukocyte count, C-reactive protein, cerebrospinal fluid leukocyte count and culture result, number of antibiotics started during admission, and empiric antibiotic therapy. Discriminatory power of this model was excellent (area under the ROC curve 0.97). The model achieved 98.8% sensitivity (95% CI 88.0% to 99.9%) and specificity of 87.9% (84.6% to 90.8%). Positive and negative predictive values were 56.9% (50.8% to 67.9%) and 99.9% (98.6% to 99.9%), respectively. Predicted yearly infection rates concurred with observed infection rates.

Conclusion

A prediction model based on multi-source data stored in a clinical data warehouse could accurately quantify rates of DRM. Automated detection using this statistical approach is feasible and could be applied to other nosocomial infections.

About the Authors:

Maaike S. M. van Mourik

* E-mail: [email protected]

Affiliation: Department of Medical Microbiology, University Medical Centre Utrecht, Utrecht, The Netherlands

Rolf H. H. Groenwold

Affiliation: Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands

Jan Willem Berkelbach van der Sprenkel

Affiliation: Department of Neurosurgery, University Medical Centre Utrecht, Utrecht, The Netherlands

Wouter W. van Solinge

Affiliation: Department of Clinical Chemistry and Hematology, University Medical Centre Utrecht, Utrecht, The Netherlands

Annet Troelstra

Affiliation: Department of Medical Microbiology, University Medical Centre Utrecht, Utrecht, The Netherlands

Marc J. M. Bonten

Affiliations Department of Medical Microbiology, University Medical Centre Utrecht, Utrecht, The Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands

Introduction

Healthcare-associated infections (HAI) pose a great burden on current medical care and are increasingly viewed as preventable complications. HAI refers to the entire scope of infections associated with medical care and also includes nosocomial infections. The European burden of HAI has been estimated at 4.5 million infections contributing to 148,000 deaths [1]. Hospitals are encouraged to report HAI rates through surveillance organizations such as the National Healthcare Safety Network (NHSN) from the Centers for Disease Control and Prevention (CDC) in the United States [2] and the PREZIES network in the Netherlands [1], [3]. Efficient registration and feedback of infection rates to healthcare workers are considered essential elements to reduce infection rates and surveillance of infection rates is increasingly demanded by policy makers and the public [4]. However, manual registration of HAI rates is time-consuming and susceptible to error due to subjective interpretation of definitions and manual data handling [5], [6].

Therefore, there is an urgent need for more efficient and reliable surveillance methods. Automated classification algorithms using data stored in electronic medical records (i.e. clinical data warehouses [7]) have been developed both at the hospital-wide and procedure-specific level with varying success, in particular for surgical site infections and (catheter-related) bloodstream infections [8]–[18]. Most models use a classification approach based on the presence of (one or more) indicators of infection, such as positive microbiology results, antibiotic use, and discharge coding. Another, less used, method is development of a multivariable model with associated cut-off values to classify patients [11], [13]. Such (automated) models are of relatively low cost, time-saving, and facilitate standardized interpretation of infection criteria [10]. However, when case-finding is based on microbiological cultures, such classification algorithms have low sensitivity for culture-negative infections and specificity of these algorithms decreases when extending case-finding criteria. Furthermore, suboptimal positive predictive values still make manual confirmation of infection necessary. Given these potential drawbacks, most healthcare centers still use manual chart review as their primary method of surveillance.

The aim of this study was to develop a prediction model for HAI using data routinely stored in a clinical data warehouse (i.e. the Utrecht patient-oriented database, UPOD [19]) to retrospectively identify the presence of infection. In order to increase sensitivity for culture-negative infections while maintaining specificity and enable surveillance without the need for manual confirmation, an alternative approach to the classification algorithm was sought and data sources were extended to include not only microbiology results and antibiotic use but also results of clinical chemistry analysis. In clinical practice, such a model could eliminate or significantly reduce the workload of manual chart review and increase resources available for development and implementation of infection control measures. Drain-related meningitis (DRM) was selected as an example to investigate this general approach to automated evaluation of infection rates. This nosocomial infection, related to external cerebrospinal fluid (CSF) drainage through external ventricular (EVD) or lumbar drains (ELD), is sometimes also termed ventriculitis or meningoventriculitis and is one of the procedure-specific infections that has since 2004 been monitored by the department of hospital hygiene and infection control through labor-intensive manual chart review. The developed model achieved good discriminatory power at the level of the individual patient and group-level estimates of infection rates could be generated without any manual confirmation.

Methods

Ethics statement

The use of anonymous data through the UPOD has been exempted from review by the Institutional Review Board of the University Medical Center Utrecht as described previously [19].

Study design and outcome measure

Data collected as part of the hospital hygiene surveillance program were used to develop the prediction model for DRM. Results of routinely performed incidence surveys were considered as reference standard. Two infection control professionals assessed each patient for the development of DRM by chart review using modified NHSN/CDC criteria for healthcare-associated meningitis (figure 1) [20], [21]; in case of disagreement adjudication was performed through review. A surveillance episode was defined to start the day of drain placement up to seven days after drain removal of the last drain or up to discharge, whichever occurred first.

[Figure omitted. See PDF.]

Figure 1. NHSN/CDC definition of healthcare-associated meningitis for patients >1 year of age [20], [21].

https://doi.org/10.1371/journal.pone.0022846.g001

Study population

All patients registered by the surveillance program to have received an external cerebrospinal fluid drain at the University Medical Centre Utrecht, a 1042-bed tertiary healthcare centre, were included in this study. Registration comprises all patients who received an EVD between January 1^st 2004 and December 31^st 2009 (with the exception of May–July 2004), and all patients receiving and ELD in 2004 to 2006. From January 2007 to December 2009 surveillance for ELD was only performed in patients who received the drain in operating theatres. Several infection control measures were implemented during the study period as described previously [20]. All EVDs are placed in operating theatres or, sometimes, under sterile conditions in the intensive care unit (ICU) by a neurosurgeon or trained resident. ELDs are either inserted in the operating theatre or in sterile conditions on the neurology ward. EVDs are tunneled five centimeters under the skin. All patients receive perioperative antibiotic prophylaxis. Drains are not exchanged on a prophylactic basis and CSF samples are collected for culture and biochemical analysis only when infection is clinically suspected; at this time empiric antibiotic therapy is initiated according to local protocol.

Throughout the study period, 742 patients received one or more drains. The following exclusion criteria were applied: death within one day after drain placement (40 patients), age under 18 years (n = 110), pre-existing central nervous system infection (n = 29), more than one simultaneous drain (n = 8), drain placement in a different centre (n = 4), admission duration less than one day (n = 3), second admission more than 30 days after discharge from a first drainage episode within the study (n = 7), and admission to the military hospital (n = 2). After application of exclusion criteria, 537 patients were available for analysis.

Data collection

The department of hospital hygiene provided outcome data along with the drain characteristics for each patient (drain type, duration, indication for placement). Prediction data was obtained through the Utrecht Patient Oriented Database (UPOD), a clinical data warehouse launched in 2004 for research purposes. The UPOD links results from laboratory analysis (clinical chemistry and hematology), microbiological cultures, and pharmacy dispensing data to information from the hospital admission and discharge system at the patient level [19]. Medication prescription data were obtained both from the UPOD as well as directly from the intensive care unit. Data were adapted to a standardized format and checked for inconsistencies. When necessary, original data sources were cross-referenced to exclude errors.

Predictor selection

Predictors were selected both on theoretical grounds and to best match the modified NHSN/CDC criteria. Duration of drainage, drain manipulation, subarachnoid hemorrhage, cerebrospinal fluid leakage, and other concomitant infections have been described as possible risk factors for meningitis previously [22]. Besides microbiological analysis of CSF and drains (after removal), biochemical markers of meningitis such as CSF leukocyte count, neutrophil count, glucose level, total protein level, and CSF/blood glucose ratio have been applied to the neurosurgical population with moderate success [23], [24] and were therefore considered as predictors. Unfortunately, Gram-stain results were not yet available and the UPOD does not contain information on drain manipulation and the occurrence of concomitant infections. If patients received both an EVD and an ELD, the EVD took priority in determining drain type. Culture results have been corrected for contamination by categorizing cultures growing coagulase-negative staphylococci as negative if no antibiotic therapy was initiated one day prior through three days after culture. Empiric antibiotic therapy was defined as the simultaneous use of vancomycine and ceftazidime (started four or more days after admission) or ceftriaxone and flucloxacillin (initiated within four days of admission) according to local protocol. The number of systemic antibiotics started throughout the surveillance episode was included as a surrogate marker for the presence of other concomitant infections.

Statistical analysis

Since the objective was to predict whether a patient had developed DRM during hospital stay, the value that was most indicative of infection measured throughout each patient's surveillance episode was taken for each predictor. Missing data were imputed using multiple imputation (ten imputations). For C-reactive protein (CRP), squared and cubic terms were included in the prediction model along with the linear term. The number of leukocytes in CSF was log-transformed prior to analyses.

Variables were selected for multivariate analysis based on theoretical considerations (as previously described) and results of univariate analysis (p<0.05 in the mean dataset). Using logistic regression analysis, a prediction model was then developed by means of manual backward selection (p<0.05). Although it is recommended to use higher p-values for selection of predictors in prediction research [25], this more stringent criterion was used due to the limited number of events (higher p-values would have resulted in too few events per predictor). Regression coefficients and standard errors were determined on each imputation set and pooled using Rubin's rule [26]. Subsequently, bootstrapping (100 samples per imputation set) was applied to correct for optimism.

Discrimination and calibration were determined for the final model. Discrimination refers to the ability to distinguish between patients with and without DRM; this was assessed by the area under the ROC curve. Calibration refers to the concordance between the predicted and observed probabilities of infection, which was assessed using a calibration plot. For clinical application, cut-off values for a predicted probability associated with high sensitivity and acceptable specificity were determined and associated sensitivity, specificity and predictive values were reported. Confidence intervals were determined using exact binomial methods. Finally, the summed predicted probabilities were used to investigate infection rates at the group level. All analyses were done using SPSS® 17 (SPSS Inc, Chicago IL) and R version 2.11.1 (www.r-project.org).

Results

A total of 691 drains were placed in 537 patients. DRM occurred in 82 patients (15.3%), or 13.5 infections per 1000 drainage days at risk. The most common causative micro-organisms were coagulase-negative staphylococci (33.8%), followed by Staphylococcus aureus (14.6%) and enterobacteriacaea (13.4%). Seventeen infections were culture negative (20.7%). Baseline characteristics are described in table 1. Median age of the included patients was 58.5 years, half (n = 263) received a CSF drainage system to treat secondary hydrocephalus following subarachnoid hemorrhage, intraventricular hemorrhage or (cerebellar) infarction and almost 60% (n = 312) of patients were admitted to the ICU during part of their stay. Patients were admitted for a median of 21 days (including readmissions within 30 days).

[Figure omitted. See PDF.]

Table 1. Baseline characteristics of the patient population after multiple imputation of missing values and univariate association between variables and the risk of drain-related meningitis.

https://doi.org/10.1371/journal.pone.0022846.t001

Based on the results of the univariate analysis, the following variables were selected for multivariate analysis: indication for drain placement, duration of admission, number of drains placed, total drainage duration, duration of ICU admission, CRP, blood leukocytes, CSF leukocytes, CSF glucose, CSF protein, culture result (CSF and/or drain), total number of antibiotics started during admission, and whether empiric antibiotic therapy for drain-related meningitis was initiated.

Table 2 shows the predictors retained in the model and their associated p-values. Despite its high p-value (p = 0.230), the linear CRP term was kept in the model in order to allow the significant high-power terms to be included. The prediction rule can be used to calculate the probability of meningitis for each patient (figure 2). Discriminatory power of the model as determined by the area under the ROC curve was 0.970 (95% CI: 0.954–0.986). Calibration of the final model was good (figure 3).

[Figure omitted. See PDF.]

Figure 2. Prediction rule for the development of drain-related meningitis.

Abbreviations: P(DRM) – probability of drain-related meningitis; LP – linear predictor; EVD – external ventricular drain; CRP – C-reactive protein; CSF – cerebrospinal fluid.

https://doi.org/10.1371/journal.pone.0022846.g002

[Figure omitted. See PDF.]

Figure 3. Calibration plot of the model predicting drain-related meningitis.

The diagonal dashed line represents ideal prediction by the model, the pointed line predicted probabilities. Calibration, or the concordance between predicted and observed probability of infection, is adequate.

https://doi.org/10.1371/journal.pone.0022846.g003

[Figure omitted. See PDF.]

Table 2. Outcome of backward stepwise logistic regression predicting the risk of drain-related meningitis.

https://doi.org/10.1371/journal.pone.0022846.t002

A cut-off in predicted probability of 0.107 resulted in 98.8% sensitivity, specificity of 87.9% and positive and negative predictive values of respectively 59.6% and 99.8% (table 3). The only missed infection was an infection with a coagulase-negative staphylococcus for which no antibiotics were started during admission and of which the patient recovered spontaneously. Selecting a cut-off probability of 0.175 missed three additional infections (sensitivity 95.1%), but only slightly improved specificity (91.0%) and positive predictive value (65.5%).

[Figure omitted. See PDF.]

Table 3. Two-by-two contingency table for predicted probability (P(DRM)) in relation to drain-related meningitis.

https://doi.org/10.1371/journal.pone.0022846.t003

If a definite diagnosis is necessary at the patient level, application of the model reduced the number of charts to review manually to 25.3% (from 537 to 136 charts) while still identifying 98.8% of infections (81 out of 82). When interested in infection rates at the group level, the summed predicted probabilities reflect total infection percentages with good concordance (figure 4) and thus allow for surveillance without the need for manual confirmation.

[Figure omitted. See PDF.]

Figure 4. Comparison of observed and predicted overall yearly infection rates.

Observed and predicted infection rates at the patient level (panel A) and expressed per 1000 drainage days at risk (panel B), including 95% confidence intervals. Predicted yearly infection rates are determined by the summed predicted probabilities and show good concordance with observed rates.

https://doi.org/10.1371/journal.pone.0022846.g004

Discussion

The results of this study show that information stored in clinical data warehouses can successfully be used to predict rates of DRM in patients receiving an external ventricular or lumbar drain. The combination of drain characteristics, microbiology and clinical chemistry results and antibiotic use achieved 98.8% sensitivity and 87.5% specificity in detecting drain-related meningitis when applying a predicted probability cut-off of 0.107. Negative and positive predictive values were 99.8% and 56.9% respectively. Performing chart review only for those patients identified by the model to have DRM would reduce the number of manual chart reviews by 74.7%. Monitoring of longitudinal infection rates at the group level, however, can be performed without manual confirmation, thereby providing an efficient surveillance tool. This study can be viewed as proof-of-concept for use of regression model-based systems to perform surveillance of nosocomial infections at the group level.

As opposed to most detection models described previously, the model presented here uses data from a multitude of sources in a multivariable model, and case-finding is based on the weighted combination of predictors from each source. As opposed to classification algorithms with case-finding based on broadly selected indicators, this weighted combination of predictors leads to high sensitivity for both culture-positive and culture-negative infections while maintaining acceptable positive predictive value. To the best of our knowledge, this is the first model also incorporating results of clinical chemistry and hematology analysis as predictors of healthcare-associated infections.

In comparison to conventional manual surveillance, this model reduces the time needed to perform surveillance, is less prone to error and less vulnerable to inter-rater variation. Furthermore, calculation of summed predicted probabilities for the at-risk population is an efficient surveillance tool to monitor changes in infection rates and determine when to perform in-depth analysis. Several studies have shown that although automated models using simplified and objective criteria may not always correctly predict absolute infection rates, such models may achieve reliable ranking of hospitals and accentuate differences between hospitals [12], [27], [28].

The large patient population included in this study allowed for the application of statistical methods as opposed to classification algorithms. Although the rule-of-thumb of ten events per predictor was violated, this does not necessarily lead to unreliable results [29]. Furthermore, the selected reference standard, the CDC/NHSN definition of healthcare-associated meningitis, has been measured consistently over time for purposes other than this research. Several other definitions of DRM have been used in literature [22], [30]–[32], however they mostly require positive culture results and therefore have low sensitivity for culture-negative infections which occurred in twenty percent of cases in this population. Even though the CDC/NHSN definition is only partially applicable to neurosurgical patients who are comatose or sedated, the other definitions of DRM will also face this problem as many require the presence of clinical symptoms to confirm the diagnosis. Although it can be argued that healthcare-associated meningitis is a different clinical entity than meningitis secondary to cerebrospinal fluid drainage, the selected reference standard has been measured consistently and reliably over the six-year period and contains many similarities to other definitions proposed for drain-related meningitis. Imputation of missing values was used to prevent the introduction of bias in deriving the model. Since it is not possible to impute missing values for individual patients, a probability of infection can not be computed for future patients with missing data. Out of the patients with an infection, only one had missing data for one predictor (CSF leukocyte count), thereby making underestimation of infection rates unlikely. Furthermore, predictors were only included if commonly determined in clinical practice. For this reason, parameters that have been described previously such as CSF lactate levels [33], [34], CSF cytokine levels [23], [35], and procalcitonin levels [36], [37] were not considered for inclusion. The calculation of the cell-index was considered as a tool to correct for blood-contaminated CSF [38]; however, since this measure could not be calculated in 65.3% of patients due to missing data, it was not included in the analyses. Finally, this model does not investigate infections occurring after discharge unless the patient is readmitted. However, contrary to surgical site infections, post-discharge surveillance is not as relevant since patients often remain in the hospital for a number of days after removal of the drain and it is customary for patients to return to their primary hospital when complications occur. These patients are then re-included in surveillance if readmission occurs within 30 days of discharge.

In summary, the model developed can accurately quantify rates of drain-related meningitis using multi-source data. The proposed model was developed using only retrospective data and although measures have been taken to prevent excessive optimism, prospective validation both within our centre and on a larger scale is necessary to assure applicability to other patient populations. This multivariable model-based approach can be applied to other types of nosocomial infections in the future. Also the development of methods to determine device utilization rates using data available through electronic healthcare records will further improve efficiency and reliability of surveillance.

Acknowledgments

The authors would like to thank all contributing data managers, H.E.M. Blok, H. den Breeijen and E. Mulder, for their contribution as well as the infection control professionals from the department of hospital hygiene and infection control (A.C.M. Gigengack-Baars, M.C.E. van der Jagt-Zwetsloot, S.M. van Dijk).

Author Contributions

Conceived and designed the experiments: MSMvM AT MJMB. Analyzed the data: MSMvM RHHG. Wrote the paper: MSMvM RHHG JWBvdS WWvS AT MJMB. Coordinated the clinical data warehouse: WWvS.

Citation: van Mourik MSM, Groenwold RHH, Berkelbach van der Sprenkel JW, van Solinge WW, Troelstra A, Bonten MJM (2011) Automated Detection of External Ventricular and Lumbar Drain-Related Meningitis Using Laboratory and Microbiology Results and Medication Data. PLoS ONE 6(8): e22846. https://doi.org/10.1371/journal.pone.0022846

References

1. ECDC (2008) Annual Epidemiological Report on Communicable Disease in Europe 2008.ECDC2008Annual Epidemiological Report on Communicable Disease in Europe 2008.Available: http://ecdc.europa.eu/en/publications/Publications/0812_SUR_Annual_Epidemiological_Report_2008.pdf. Accessed 2011 Feb 23. Available: http://ecdc.europa.eu/en/publications/Publications/0812_SUR_Annual_Epidemiological_Report_2008.pdf. Accessed 2011 Feb 23.

2. Edwards JR, Peterson KD, Mu Y, Banerjee S, len-Bridson K, et al. (2009) National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009. Am J Infect Control 37: 783–805.JR EdwardsKD PetersonY. MuS. BanerjeeK. len-Bridson2009National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009.Am J Infect Control37783805

3. van der Kooi TI, Mannien J, Wille JC, van Benthem BH (2010) Prevalence of nosocomial infections in The Netherlands, 2007–2008: results of the first four national studies. J Hosp Infect 75: 168–72.TI van der KooiJ. MannienJC WilleBH van Benthem2010Prevalence of nosocomial infections in The Netherlands, 2007–2008: results of the first four national studies.J Hosp Infect7516872

4. Haley RW, Culver DH, White JW, Morgan WM, Emori TG, et al. (1985) The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals. Am J Epidemiol 121: 182–205.RW HaleyDH CulverJW WhiteWM MorganTG Emori1985The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals.Am J Epidemiol121182205

5. Ehrenkranz NJ, Shultz JM, Richter EL (1995) Recorded criteria as a “gold standard” for sensitivity and specificity estimates of surveillance of nosocomial infection: a novel method to measure job performance. Infect Control Hosp Epidemiol 16: 697–702.NJ EhrenkranzJM ShultzEL Richter1995Recorded criteria as a “gold standard” for sensitivity and specificity estimates of surveillance of nosocomial infection: a novel method to measure job performance.Infect Control Hosp Epidemiol16697702

6. Gastmeier P, Kampf G, Hauer T, Schlingmann J, Schumacher M, et al. (1998) Experience with two validation methods in a prevalence survey on nosocomial infections. Infect Control Hosp Epidemiol 19: 668–73.P. GastmeierG. KampfT. HauerJ. SchlingmannM. Schumacher1998Experience with two validation methods in a prevalence survey on nosocomial infections.Infect Control Hosp Epidemiol1966873

7. Trick WE (2008) Building a data warehouse for infection control. Am J Infect Control 36: S75–81.WE Trick2008Building a data warehouse for infection control.Am J Infect Control36S7581

8. Bellini C, Petignat C, Francioli P, Wenger A, Bille J, et al. (2007) Comparison of automated strategies for surveillance of nosocomial bacteremia. Infect Control Hosp Epidemiol 28: 1030–5.C. BelliniC. PetignatP. FrancioliA. WengerJ. Bille2007Comparison of automated strategies for surveillance of nosocomial bacteremia.Infect Control Hosp Epidemiol2810305

9. Woeltje KF, Butler AM, Goris AJ, Tutlam NT, Doherty JA, et al. (2008) Automated surveillance for central line-associated bloodstream infection in intensive care units. Infect Control Hosp Epidemiol 29: 842–6.KF WoeltjeAM ButlerAJ GorisNT TutlamJA Doherty2008Automated surveillance for central line-associated bloodstream infection in intensive care units.Infect Control Hosp Epidemiol298426

10. Klompas M, Yokoe DS (2009) Automated surveillance of health care-associated infections. Clin Infect Dis 48: 1268–75.M. KlompasDS Yokoe2009Automated surveillance of health care-associated infections.Clin Infect Dis48126875

11. Yokoe DS, Noskin GA, Cunnigham SM, Zuccotti G, Plaskett T, et al. (2004) Enhanced identification of postoperative infections among inpatients. Emerg Infect Dis 10: 1924–30.DS YokoeGA NoskinSM CunnighamG. ZuccottiT. Plaskett2004Enhanced identification of postoperative infections among inpatients.Emerg Infect Dis10192430

12. Platt R, Kleinman K, Thompson K, Dokholyan RS, Livingston JM, et al. (2002) Using automated health plan data to assess infection risk from coronary artery bypass surgery. Emerg Infect Dis 8: 1433–41.R. PlattK. KleinmanK. ThompsonRS DokholyanJM Livingston2002Using automated health plan data to assess infection risk from coronary artery bypass surgery.Emerg Infect Dis8143341

13. Sands K, Vineyard G, Livingston J, Christiansen C, Platt R (1999) Efficient identification of postdischarge surgical site infections: use of automated pharmacy dispensing information, administrative data, and medical record information. J Infect Dis 179: 434–41.K. SandsG. VineyardJ. LivingstonC. ChristiansenR. Platt1999Efficient identification of postdischarge surgical site infections: use of automated pharmacy dispensing information, administrative data, and medical record information.J Infect Dis17943441

14. Bolon MK, Hooper D, Stevenson KB, Greenbaum M, Olsen MA, et al. (2009) Improved surveillance for surgical site infections after orthopedic implantation procedures: extending applications for automated data. Clin Infect Dis 48: 1223–9.MK BolonD. HooperKB StevensonM. GreenbaumMA Olsen2009Improved surveillance for surgical site infections after orthopedic implantation procedures: extending applications for automated data.Clin Infect Dis4812239

15. Graham PL III, San GP, Lutwick S, Haas J, Saiman L (2004) Validation of a multicenter computer-based surveillance system for hospital-acquired bloodstream infections in neonatal intensive care departments. Am J Infect Control 32: 232–4.PL Graham IIIGP SanS. LutwickJ. HaasL. Saiman2004Validation of a multicenter computer-based surveillance system for hospital-acquired bloodstream infections in neonatal intensive care departments.Am J Infect Control322324

16. Trick WE, Zagorski BM, Tokars JI, Vernon MO, Welbel SF, et al. (2004) Computer algorithms to detect bloodstream infections. Emerg Infect Dis 10: 1612–20.WE TrickBM ZagorskiJI TokarsMO VernonSF Welbel2004Computer algorithms to detect bloodstream infections.Emerg Infect Dis10161220

17. Pokorny L, Rovira A, Martin-Baranera M, Gimeno C, onso-Tarres C, et al. (2006) Automatic detection of patients with nosocomial infection by a computer-based surveillance system: a validation study in a general hospital. Infect Control Hosp Epidemiol 27: 500–3.L. PokornyA. RoviraM. Martin-BaraneraC. GimenoC. onso-Tarres2006Automatic detection of patients with nosocomial infection by a computer-based surveillance system: a validation study in a general hospital.Infect Control Hosp Epidemiol275003

18. Brossette SE, Hacek DM, Gavin PJ, Kamdar MA, Gadbois KD, et al. (2006) A laboratory-based, hospital-wide, electronic marker for nosocomial infection: the future of infection control surveillance? Am J Clin Pathol 125: 34–9.SE BrossetteDM HacekPJ GavinMA KamdarKD Gadbois2006A laboratory-based, hospital-wide, electronic marker for nosocomial infection: the future of infection control surveillance?Am J Clin Pathol125349

19. ten Berg MJ, Huisman A, van den Bemt PM, Schobben AF, Egberts AC, et al. (2007) Linking laboratory and medication data: new opportunities for pharmacoepidemiological research. Clin Chem Lab Med 45: 13–9.MJ ten BergA. HuismanPM van den BemtAF SchobbenAC Egberts2007Linking laboratory and medication data: new opportunities for pharmacoepidemiological research.Clin Chem Lab Med45139

20. Leverstein-Van Hall MA, Hopmans TEM, Van Der Sprenkel JWB, Blok HEM, Van Der Mark WAMA, et al. (2010) A bundle approach to reduce the incidence of external ventricular and lumbar drain-related infections: Clinical article. J Neurosurg 112: 345–53.MA Leverstein-Van HallTEM HopmansJWB Van Der SprenkelHEM BlokWAMA Van Der Mark2010A bundle approach to reduce the incidence of external ventricular and lumbar drain-related infections: Clinical article.J Neurosurg11234553

21. Horan TC, Andrus M, Dudeck MA (2008) CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control 36: 309–32.TC HoranM. AndrusMA Dudeck2008CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting.Am J Infect Control3630932

22. Lozier AP, Sciacca RR, Romagnoli MF, Connolly ES Jr (2002) Ventriculostomy-related infections: a critical review of the literature. Neurosurgery 51: 170–81.AP LozierRR SciaccaMF RomagnoliES Connolly Jr2002Ventriculostomy-related infections: a critical review of the literature.Neurosurgery5117081

23. Schade RP, Schinkel J, Roelandse FW, Geskus RB, Visser LG, et al. (2006) Lack of value of routine analysis of cerebrospinal fluid for prediction and diagnosis of external drainage-related bacterial meningitis. J Neurosurg 104: 101–8.RP SchadeJ. SchinkelFW RoelandseRB GeskusLG Visser2006Lack of value of routine analysis of cerebrospinal fluid for prediction and diagnosis of external drainage-related bacterial meningitis.J Neurosurg1041018

24. Pfisterer W, Muhlbauer M, Czech T, Reinprecht A (2003) Early diagnosis of external ventricular drainage infection: results of a prospective study. J Neurol Neurosurg Psychiatry 74: 929–32.W. PfistererM. MuhlbauerT. CzechA. Reinprecht2003Early diagnosis of external ventricular drainage infection: results of a prospective study.J Neurol Neurosurg Psychiatry7492932

25. Steyerberg EW (2008) Clinical Prediction Models: A practical approach to development, validation, and updating. New York: Springer. 497 p.EW Steyerberg2008Clinical Prediction Models: A practical approach to development, validation, and updatingNew YorkSpringer497

26. Rubin DB (1987) Multiple Imputation for Nonresponse in Surveys. Hoboken: J. Wiley & Sons. 288 p.DB Rubin1987Multiple Imputation for Nonresponse in SurveysHobokenJ. Wiley & Sons288

27. Huang SS, Livingston JM, Rawson NS, Schmaltz S, Platt R (2007) Developing algorithms for healthcare insurers to systematically monitor surgical site infection rates. BMC Med Res Methodol 7: 20.SS HuangJM LivingstonNS RawsonS. SchmaltzR. Platt2007Developing algorithms for healthcare insurers to systematically monitor surgical site infection rates.BMC Med Res Methodol720

28. Rubin MA, Mayer J, Greene T, Sauer BC, Hota B, et al. (2008) An agent-based model for evaluating surveillance methods for catheter-related bloodstream infection. AMIA Annu Symp Proc 631–5.MA RubinJ. MayerT. GreeneBC SauerB. Hota2008An agent-based model for evaluating surveillance methods for catheter-related bloodstream infection.AMIA Annu Symp Proc6315

29. Vittinghoff E, McCulloch CE (2007) Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 165: 710–8.E. VittinghoffCE McCulloch2007Relaxing the rule of ten events per variable in logistic and Cox regression.Am J Epidemiol1657108

30. Mayhall CG, Archer NH, Lamb VA, Spadora AC, Baggett JW, et al. (1984) Ventriculostomy-related infections. A prospective epidemiologic study. N Engl J Med 310: 553–9.CG MayhallNH ArcherVA LambAC SpadoraJW Baggett1984Ventriculostomy-related infections. A prospective epidemiologic study.N Engl J Med3105539

31. Sundbarg G, Nordstrom CH, Soderstrom S (1988) Complications due to prolonged ventricular fluid pressure recording. Br J Neurosurg 2: 485–95.G. SundbargCH NordstromS. Soderstrom1988Complications due to prolonged ventricular fluid pressure recording.Br J Neurosurg248595

32. Holloway KL, Barnes T, Choi S, Bullock R, Marshall LF, et al. (1996) Ventriculostomy infections: the effect of monitoring duration and catheter exchange in 584 patients. J Neurosurg 85: 419–24.KL HollowayT. BarnesS. ChoiR. BullockLF Marshall1996Ventriculostomy infections: the effect of monitoring duration and catheter exchange in 584 patients.J Neurosurg8541924

33. Leib SL, Boscacci R, Gratzl O, Zimmerli W (1999) Predictive value of cerebrospinal fluid (CSF) lactate level versus CSF/blood glucose ratio for the diagnosis of bacterial meningitis following neurosurgery. Clin Infect Dis 29: 69–74.SL LeibR. BoscacciO. GratzlW. Zimmerli1999Predictive value of cerebrospinal fluid (CSF) lactate level versus CSF/blood glucose ratio for the diagnosis of bacterial meningitis following neurosurgery.Clin Infect Dis296974

34. Wong GK, Poon WS, Ip M (2008) Use of ventricular cerebrospinal fluid lactate measurement to diagnose cerebrospinal fluid infection in patients with intraventricular haemorrhage. J Clin Neurosci 15: 654–5.GK WongWS PoonM. Ip2008Use of ventricular cerebrospinal fluid lactate measurement to diagnose cerebrospinal fluid infection in patients with intraventricular haemorrhage.J Clin Neurosci156545

35. Lopez-Cortes LF, Marquez-Arbizu R, Jimenez-Jimenez LM, Jimenez-Mejias E, Caballero-Granado FJ, et al. (2000) Cerebrospinal fluid tumor necrosis factor-alpha, interleukin-1beta, interleukin-6, and interleukin-8 as diagnostic markers of cerebrospinal fluid infection in neurosurgical patients. Crit Care Med 28: 215–9.LF Lopez-CortesR. Marquez-ArbizuLM Jimenez-JimenezE. Jimenez-MejiasFJ Caballero-Granado2000Cerebrospinal fluid tumor necrosis factor-alpha, interleukin-1beta, interleukin-6, and interleukin-8 as diagnostic markers of cerebrospinal fluid infection in neurosurgical patients.Crit Care Med282159

36. Berger C, Schwarz S, Schaebitz WR, Aschoff A, Schwab S (2002) Serum procalcitonin in cerebral ventriculitis. Crit Care Med 30: 1778–81.C. BergerS. SchwarzWR SchaebitzA. AschoffS. Schwab2002Serum procalcitonin in cerebral ventriculitis.Crit Care Med30177881

37. Martinez R, Gaul C, Buchfelder M, Erbguth F, Tschaikowsky K (2002) Serum procalcitonin monitoring for differential diagnosis of ventriculitis in adult intensive care patients. Intensive Care Med 28: 208–10.R. MartinezC. GaulM. BuchfelderF. ErbguthK. Tschaikowsky2002Serum procalcitonin monitoring for differential diagnosis of ventriculitis in adult intensive care patients.Intensive Care Med2820810

38. Pfausler B, Beer R, Engelhardt K, Kemmler G, Mohsenipour I, et al. (2004) Cell index–a new parameter for the early diagnosis of ventriculostomy (external ventricular drainage)-related ventriculitis in patients with intraventricular hemorrhage? Acta Neurochir (Wien) 146: 477–81.B. PfauslerR. BeerK. EngelhardtG. KemmlerI. Mohsenipour2004Cell index–a new parameter for the early diagnosis of ventriculostomy (external ventricular drainage)-related ventriculitis in patients with intraventricular hemorrhage?Acta Neurochir (Wien)14647781