Validation of the Prediction of Delirium for Intensive Care model to predict subsyndromal delirium

Abstract

Aim

Subsyndromal delirium is associated with prolonged intensive care unit stays, and prolonged mechanical ventilation requirements. The Prediction of Delirium for Intensive Care (PRE‐DELIRIC) model can predict delirium. This study was designed to verify if it can also predict development of subsyndromal delirium.

Methods

We undertook a single‐center, retrospective observation study in Japan. We diagnosed subsyndromal delirium based on the Intensive Care Delirium Screening Checklist. We calculated the sensitivity and specificity of the PRE‐DELIRIC model and obtained a diagnostic cut‐off value.

Results

We evaluated data from 70 patients admitted to the mixed medical intensive care unit of the Tokyo Medical University Hospital (Tokyo, Japan) between May 2015 and February 2017. The prevalence of subsyndromal delirium by Intensive Care Delirium Screening Checklist was 31.4%. The area under the receiver operating characteristic curve was 0.83 of the PRE‐DELIRIC model for subsyndromal delirium. The calculated cut‐off value was 36 points with a sensitivity of 94.3% and specificity of 57.1%. Subsyndromal delirium was associated with a higher incidence of delirium (odds ratio, 8.81; P < 0.01).

Conclusion

The PRE‐DELIRIC model could be a tool for predicting subsyndromal delirium using a cut‐off value of 36 points.

Introduction

Delirium is a clinical syndrome of acute cerebral dysfunction characterized by three cardinal features: fluctuating mental status, inattention, and altered level of consciousness or disorganized thinking. Delirium develops with changes in environmental and physical conditions, and it has a prevalence of 3–56% among hospitalized patients. The risk of delirium is elevated in elderly and physically weak individuals, such as patients in the intensive care unit (ICU) and those receiving mechanical ventilation. Patients developing delirium have prolonged stays in the ICU and hospital, with associated higher in‐hospital and overall mortality rates and a higher incidence of long‐term cognitive dysfunction.

Subsyndromal delirium (SSD) is defined as a condition that does not satisfy the diagnostic delirium criteria of the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM‐5), but that has one or more symptoms characteristic of delirium. Subsyndromal delirium is present when the Intensive Care Delirium Screening Checklist (ICDSC) is between 1 and 3. Subsyndromal delirium in the ICU is a cause of increased mortality and a prolonged stay. At the same time, SSD itself has been identified as a risk factor for development of delirium.

The pathophysiologic mechanisms underlying delirium, albeit still unclear, are related to imbalances in neurotransmitters modulating cognition, behavior, and mood. The average medical ICU patient has 11 or more risk factors for developing delirium, which can be divided into predisposing baseline factors such as underlying patient characteristics and comorbidities and hospital‐related (precipitating) factors such as acute illness, treatment, and ICU management. Prediction of Delirium for Intensive Care (PRE‐DELIRIC) is a model for predicting the incidence of delirium in adult ICU inpatients. The model consists of 10 risk factors (age, Acute Physiology and Chronic Health Assessment II [APACHE‐II] score, hospitalization, coma, infection, metabolic acidosis, sedative drug use, morphine use, urea concentration, and emergency hospitalization) that can be evaluated within 24 h after ICU admission. Its predictive ability was evaluated by an area under the receiver operating characteristic (ROC) curve at 0.87 (95% confidence interval, 0.85–0.89). The PRE‐DELIRIC model is useful as a quantitative evaluation tool for the risk of developing delirium at hospitalization.

Subsyndromal delirium is associated with a bad prognosis for ICU inpatients and is itself a risk for delirium development. Therefore, predicting the development of SSD in the ICU would be useful for management. In this study, we sought to determine cut‐off values for PRE‐DELIRIC to predict the occurrence of delirium and SSD.

Method

Participants

This study was carried out between May 2015 and February 2017 at the mixed medical ICU of the Tokyo Medical University Hospital (Tokyo, Japan). The ethical committee at the Tokyo Medical University approved the study protocol, and we obtained written informed consent from all participants or their surrogate health‐care decision‐makers. Patients older than 20 years and admitted to the ICU for <24 h were considered eligible. We excluded patients if they had a life expectancy of <48 h or had baseline dementia, severe liver dysfunction, or a history of alcohol or other substance abuse, or if they had received medications for delirium before their admission to the ICU (e.g., antipsychotics, antidepressants, and hypnotics).

We set the observation period for our study at 7 days during the ICU stay. We excluded patients with cognitive dysfunction, severe liver dysfunction, and disturbance of consciousness because it is difficult to distinguish delirium from these diseases in such a short time (7 days). We also excluded patients with a history of alcoholism or psychotropic dependence that might have had a condition similar to delirium due to withdrawal syndrome. Finally, patients received treatment according to the ABCDEF bundle (awakening and breathing coordination, choice of drugs, delirium monitoring and management, early mobility, and family engagement) as a non‐pharmacological approach to delirium. The ABCDEF bundle is a non‐pharmacological intervention shown to be effective both in the prevention and in the treatment of ICU delirium. It is used to ensure optimal pain management, avoid deep sedation, avoid benzodiazepine use (avoid use of hair‐loss‐inducing drug therapies), and accelerate mechanical ventilation withdrawal. The interventions promote removal of unnecessary catheters and tubes from the patient, avoid using constraints, promote normal sleep patterns, and help reorient and mobilize patients. These non‐pharmacological ICU delirium management strategies were designed to address the patient's ICU deliberate modifiable risk factors. In the study by Barnes‐Daly et al., application of the ABCDEF bundle halved the incidence of delirium in ICU.

Measurements

Intensive care nurses were trained to evaluate the onset of delirium in patients using the ICDSC. Subsequently, patients with an ICDSC score ≥1 were evaluated by a psychiatrist to confirm the diagnosis of delirium using DSM‐5. When the ICDSC score increased to ≥1, psychiatrists were immediately briefed and a psychiatrist consultation was provided to the patient. Psychiatrists were always assigned to the ICU during the study period. Thus, such patients were evaluated using DSM‐5 on the same day when ICDSC score increased to ≥1. Data on patient characteristics were collected at admission, and the patients’ conditions were scored using the sequential organ failure assessment score and the APACHE II score. The baseline risk of delirium was assessed using the PRE‐DELERIC delirium risk score, based on blood test and vital sign data from admission examination.

Outcomes

The primary outcome was to calculate the cut‐off value at which PRE‐DELIRIC predicts ICDSC score ≥1. Trained nurses screened the onset of cognitive decline using the ICDSC at the same time every morning (10.00–11.00 am) and defined the ICDSC cut‐off score as 1 point to detect SSD as a precursor of delirium.

The secondary outcome was to calculate the cut‐off value at which the PRE‐DELIRIC model predicts ICDSC ≥4 and DSM‐5. The sensitivity and specificity of ICDSC and a delirium diagnosis of DSM‐5 were calculated. As an association of SSD and delirium, patients with ICDSC 1–3 were identified as those belonging to the SSD group. Furthermore, we identified patients who developed from SSD to delirium and compared the incidence of delirium between SSD and non‐SSD groups. The SSD group included patients who had an ICDSC score of 1–3 at least once during the observation period. For example, patients with an ICDSC score of 2 on day 1 and ICDSC score of 5 on day 2 were included in the SSD group. This helped identify patients who developed delirium after the diagnosis of SSD. In addition, patients with ICDSC score of 1–3 after ICDSC score ≥4 were excluded from the SSD group because these patients had already been diagnosed with delirium. The non‐SSD group included non‐SSD patients.

Statistical analysis

We calculated the cut‐off value of PRE‐DELIRIC using the Japanese ICDSC as a gold standard and obtained the sensitivity, specificity, positive predictive, and negative predictive values to measure the adequacy of the delirium evaluation. In addition, we drew the ROC curve and selected the cut‐off value. The Mann–Whitney U‐test, χ²‐test, or Fisher's exact test were used for comparing continuous and categorical variables, respectively. We analyzed all data using the spss software (version 24; IBM, Armonk, NY, USA) and considered differences as statistically significant when P‐values were lower than 0.05.

Results

Figure  shows the flow chart for our survey. During the study period, 1,526 patients were admitted to the ICU. Of these, we excluded 1,456 from the study because they did not meet the inclusion criteria (n = 1,299) or their ICDSC data were inadequate (n = 157). Finally, 70 ICU patients were considered eligible and participated in the study. The baseline demographics are shown in Table . All patients were initially screened for SSD using the ICDSC tool. We found ICDSC ≥1 in 22 cases (31.4%). The average PRE‐DELIRIC value was 39.6 ± 28.6. From this result, we drew the ROC curves (Fig. A), and calculated sensitivity and specificity. With a cut‐off of 36 points, the sensitivity was 90.9%, specificity was 70.8%, positive predictive value was 58.8%, and negative predictive value was 94.4% (Table ).

Participants’ baseline demographic and clinical characteristics

	n = 70
Age at enrolment, years	61.7 ± 20.1
Gender, male/female	54/16 (77.1)
Hypertension	21 (30.0)
Diabetes mellitus	11 (15.7)
Smoker	22 (31.4)
Admission diagnosis
Trauma	35 (50.0)
Respiratory failure	8 (11.4)
Infection	16 (22.9)
Heart disease	7 (10.0)
Others	4 (5.7)
APACHE II score	11.1 ± 7.5
PRE‐DELERIC	39.6 ± 28.6
RASS	0.01 ± 1.19
Length of ICU stay, days	8.8 ± 9.5
Use of ventilator	36 (51.4)
Time on ventilation, days	2.1 ± 2.6
Total fentanyl dose, mg	3.1 ± 4.7
Total dexmedetomidine dose, μg	1,200 ± 1,526
Total propofol dose, mg	961 ± 1,093
Incidence of ICDSC ≥1	22 (31.4)
Incidence of ICDSC ≥4	14 (20.0)
Incidence of delirium by DSM‐5	17 (21.4)

Data are reported as mean ± standard deviation or n (%). APACHE II, Acute Physiology and Chronic Health Evaluation II; DSM‐5, Diagnostic and Statistical Manual of Mental Disorders, 5th edition; ICDSC, Intensive Care Delirium Screening Checklist; ICU, intensive care unit; PRE‐DELERIC, Prediction of Delirium for Intensive Care; RASS, Richmond Agitation–Sedation Scale.

Cut‐off value, sensitivity, specificity, positive predictive value, and negative predictive value for the Prediction of Delirium for Intensive Care (PRE‐DELIRIC) model

	PRE‐DELIRIC cut‐off	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)
ICDSC ≥1	36	94.3	57.1	68.8	90.9
ICDSC ≥4	45	92.9	73.2	46.4	97.6
DSM‐5	40	88.2	67.9	46.9	94.7

DSM‐5, Diagnostic and Statistical Manual of Mental Disorders, 5th edition; ICDSC, Intensive Care Delirium Screening Checklist.

We found ICDSC ≥4 in 14 cases (20.0%). We drew the ROC curves (Fig. B), and calculated sensitivity and specificity. With a cut‐off of 45 points, the sensitivity was 92.9%, specificity was 75.0%, positive predictive value was 48.1%, and negative predictive value was 97.7% (Table ).

We found that this was diagnosed as delirium by DSM‐5 in 17 cases (24.2%). We drew the ROC curves (Fig. C) and calculated the sensitivity and specificity. The ROC area was 0.82. With a cut‐off of 40 points, the sensitivity was 88.2%, specificity was 67.9%, positive predictive value was 46.9%, and negative predictive value was 94.7% (Table ).

The sensitivity and specificity of ICDSC ≥4 for delirium diagnosis of DSM‐5 were calculated. With a cut‐off of 4 points, the sensitivity was 76.5% and specificity was 98.1%.

We identified a total of 15 patients in the SSD group (15/70, 21.4%). In the SSD group, seven patients with an ICDSC score ≥4, following an ICDSC score of 1–3, were included. Additionally, nine cases diagnosed with delirium by DSM‐5, following a diagnosis of SSD (9/15, 60%) were included in the SSD group. In total, 55 patients were included in the non‐SSD group (55/70, 78.6%), of which eight patients were diagnosed with delirium by DSM‐5 (8/55, 14.5%). The SSD group had a significantly higher incidence of delirium than the control group (odds ratio, 8.81; P < 0.01).

Discussion

In this study, we examined the SSD prediction power of the PRE‐DELIRIC test. To our knowledge, this is the first SSD prediction study for general intensive care patients.

We included patients admitted to the ICU, including those on mechanical ventilators. The severity of their condition was evaluated by APACHE II and was on average 11.1 ± 7.5. The number of patients on mechanical ventilation was 36 (51.6%). Compared with the PRE‐DELIRIC study reported by van den Boogaard et al., our APACHE II values were similar to those in that study, but the percentage of patients on mechanical ventilation was lower in our study (APACHE II, 15; ventilator use, 83.3%). The average value of the Richmond Agitation–Sedation Scale was 0.01 ± 1.19. As this was within the target range shown in the Pain, Agitation, and Delirium guidelines, we consider that appropriate analgesia and sedation were achieved. We used an average of 3.1 ± 4.7 mg fentanyl during the ventilation management, and 961 ± 1,093 mg propofol or 1,200 ± 1,526 μg dexmedetomidine for sedation (we did not use midazolam). These were regarded as specifications similar to those shown in the Pain, Agitation, and Delirium guidelines.

The diagnostic criteria for delirium are based on attention disorder, cognitive function impairment, and cognitive dysfunction variability. Subsyndromal delirium does not satisfy all of these diagnostic criteria, but it encompasses a part of them. However, the definition of SSD has not been standardized and, in general, SSD represents an intermediate stage between delirium and normal mental status. Cole et al. stated that the factors leading to SSD and those leading to delirium are the same. Therefore, we used the PRE‐DELIRIC effectively for predicting the onset of SSD.

The pathophysiology of delirium is thought to be associated with sleep disturbances in critically ill patients. Sleep hygiene programs in ICUs improve sleep quality and quantity and can reduce the incidence of delirium. Attention disturbance, a core delirium symptom, is among the most important symptoms and a critical item in the diagnostic criteria for delirium based on DSM‐5. Attention disturbances are caused by dysfunction in the ascending reticular activating system, which regulates arousal. Arousal maintenance is controlled by multiple neurotransmitters (acetylcholine, dopamine, serotonin, histamine, and noradrenaline) released from neurons in the ascending reticular activating system and brainstem. Hatta et al. have reported that the orexin receptor antagonist, suvorexant, is effective for the prevention of delirium. Furthermore, de Lecea et al. reported that orexin‐containing neurons are powerful orchestrators of various neurotransmitters involved in the sleep/wake cycle dynamics. Improvement of sleep/waking cycle dynamics could improve attention disorders and reduce delirium development.

Subsyndromal delirium is itself a risk factor for delirium onset. Shim et al. showed that patients with SSD were 1.07–8.37 times more likely to develop delirium the next day than patients without SSD. In this study, patients diagnosed with SSD had a risk of developing delirium with an odds ratio of 8.81. Therefore, SSD treatment could help prevent delirium before its development. Although SSD has a similar pathophysiology to that of delirium with attention disorder, it could be a state with few noticeable symptoms. Patients with SSD should receive delirium prevention, such as improvement of sleep/waking cycle dynamics. Antipsychotics are widely used to treat delirium in common clinical situations. However, they are associated with severe side‐effects such as increased cardiovascular events and risk of death. Symptoms of SSD include excitement, emotional variability, nightmares, and delusion, and it seems that treatment of the sleep/wake cycle dynamics with suvorexant is as effective as antipsychotic agents without their serious side‐effects. However, these findings need further verification by clinical research.

In this study, with a cut‐off value of 36 points for the PRE‐DELIRIC test, the sensitivity was 90.9%, specificity was 70.8%, positive predictive value was 58.8%, and negative predictive value was 94.4%. A high negative predictive value is required to be able to use PRE‐DELIRIC for screening patients with high delirium risks on admission. Van den Boogaard et al. introduced an operation method in which the PRE‐DELIRIC cut‐off is set to 50. However, we propose to reduce this cut‐off value to 36 points to allow for SSD risk assessment as well. Further studies need to confirm whether interventions based on this cut‐off value help to prevent the development of delirium.

We are aware of the limitations in this study. First, this was a single‐center, retrospective study. Second, we excluded many patients (mainly because many were coma patients but also patients with dementia or alcohol addictions) who are commonly present in the ICU. Finally, we failed to retrieve data on sleep patterns or on the environment (although environmental differences were unlikely due to all cases being in the same hospital).

Conclusions

The PRE‐DELIRIC model was able to predict SSD in ICU patients. We recommend setting the PRE‐DELIRIC cut‐off value to 36 points, making it is easier to set SSD prevention goals. Subsequently, it might be possible to standardize delirium prevention.

Disclosure

Approval of the research protocol: The ethical committee at the Tokyo Medical University approved the study protocol (approval no. 2954).

Informed consent: All participants provided written informed consent.

Animal studies: N/A.

Conflict of interest: None.