Evaluation of the Allergy Fact Checker, a clinical decision support system for non-invasive beta-lactam delabeling: a mixed-methods study

Abstract

Penicillin allergy delabeling strategies are time- and resource-consuming. In this mixed-methods study, we evaluated the ‘Allergy Fact Checker’, a novel clinical decision support system designed to identify patients with uneventful re-exposure to penicillins, enabling delabeling without additional testing. The intervention involved implementation of the tool, pharmacist-led review of alerts, and physician contact. An interrupted time series analysis assessed delabeling outcomes, and physicians were interviewed to explore implementation barriers. The Allergy Fact Checker identified 8.9% (164/1850) of patients with a penicillin allergy label. The delabeling rate increased from 12% to 60% (OR 6.8 (95% CI 1.3–35, P = 0.02)). Addressing one alert took 5 min ( ± 3 min). Reported barriers included workload and limited knowledge of the allergy label removal process. By streamlining the identification process, the Allergy Fact Checker offers a low-resource, scalable solution that significantly improves delabeling and integrates easily into existing clinical workflows.

Introduction

Antibiotic allergy, particularly penicillin allergy, is frequently documented in the electronic patient record (EPR), with percentages ranging from 0.9–7% in Europe to 10–20% in the United States of America and Australia¹. Over the last decade, research has shown that over 95% of these allergy labels are incorrect^{2, 3, 4, 5–6}. This large proportion of incorrect labels is problematic as it complicates treatment choices and is associated with higher overall costs, more frequent use of broad-spectrum antibiotics, increased rate of infections with Clostridioides difficile and multi-resistant organisms such as methicillin-resistant Staphylococcus aureus and vancomycin-resistant Enterococcus, increased length-of-hospital-stay and increased morbidity and mortality^{3, 4, 5, 6, 7–8}.

Recognition of these risks has led to implementing delabeling strategies and protocols, as promoted in the Infectious Disease Society of America (IDSA) antimicrobial stewardship guidelines in 2016⁹. Recently, more guidelines have been published on penicillin allergy delabeling^{10, 11–12}. Allergy workups, including skin and provocation testing, are the standard approach to invalidate incorrect labels but are invasive, time- and resource-consuming. Consequently, studies were set up to evaluate direct oral challenges in low-risk patients as a more rapid and cost-effective way of allergy delabeling^{13, 14, 15–16}. However, this method is limited to low-risk patients and remains time- and resource-consuming as it necessitates medical supervision, monitoring of vital signs, and availability of emergency medication.

To address these limitations, our research group evaluated a strictly non-invasive delabeling protocol. Non-invasive delabeling is based on clinical history alone, without the need for provocation testing^{16, 17, 18–19}. It typically includes patient questionnaires, thorough EPR reviews and contacting primary care healthcare providers. We demonstrated the potential of such a non-invasive protocol in Belgium with the successful delabeling of 40% of penicillin allergy labels²⁰. Despite being strictly non-invasive, our intervention remained time-consuming due to the manual review of the EPR of eligible patients, i.e., all patients with a penicillin allergy label²⁰. This led to the hypothesis that the development of a semi-automated clinical decision support system, designed to identify those patients with documented re-exposure without allergic reaction, could enhance the efficiency of our approach.

This hypothesis was based on the observation that many patients received penicillins in daily clinical practice despite having a penicillin allergy label in their EPR^{16, 17–18,20}. In most cases, however, uneventful re-exposure did not result in the removal of the allergy label²⁰. This is highly inefficient, as the allergy label is often reconsidered as valid during subsequent hospitalizations, leading to repeated label reviews, delays in appropriate antibiotic therapy, and the use of suboptimal antibiotics^{3, 4, 5, 6, 7–8}.

Therefore, we aimed to develop a clinical decision support system, called the ‘Allergy Fact Checker’, and evaluate its impact on delabeling in an interrupted time series (ITS) analysis. Furthermore, we aimed to qualitatively explore potential barriers to delabeling encountered by physicians when implementing this intervention in clinical practice.

Results

Development and implementation of the Allergy Fact Checker

The Allergy Fact Checker was incorporated into the hospital information system as an ‘if-then’ algorithm, using real-time structured data available in the EPR (Table 1). Based on the results of our previous study²⁰, the Allergy Fact Checker was designed to screen for patients with in-hospital administrations of benzylpenicillin, amoxicillin or amoxicillin clavulanic acid after registration of an unspecified penicillin allergy^10,12,21. In addition, it lists data on co-administration of antihistamines, glucocorticoids or epinephrin during the re-exposure period, up to 24 h after the last antibiotic dose. These medications were considered proxies for possible allergic reactions and helped inform the subsequent review process. The results of the screening, i.e., the clinical rule alerts, were compiled on a structured worklist in the hospital information system for secondary review by a specified trained person, i.e., a clinical pharmacist in our study, before alerting the treating physician²². The pharmacist conducted a manual EPR to determine whether delabeling was appropriate. If deemed appropriate, a recommendation was provided in the EPR addressing the treating physician, and this was done once per hospital admission. Patients were deemed ineligible for direct delabeling if the allergy label specified that the culprit drug was flucloxacillin, piperacillin, or temocillin, or when EPR review indicated that a reaction occurred during or after re-exposure (e.g., recorded in free-text notes). In case where the documented index reaction was very severe (i.e., anaphylactic shock or severe cutaneous adverse reactions (SCARs)), an allergy specialist was consulted before a delabeling recommendation was sent. A validated flowchart was developed to support the review, ensuring a standardized and systematic approach (Supplementary Fig. 1).

Table 1. Implemented algorithm with details of clinical rule alert criteria and source

		Source
Alert criterion 1	Registration of an unspecified penicillin allergy label	Allergy registration module
Alert criterion 2	Administration of benzylpenicillin OR amoxicillin OR amoxicillin clavulanic acid in the period after allergy registration up until the current day (‘re-exposure’)	Computerized physician order entry
Alert criterion 3	Age ≥ 18 years	Demographic data
Additional information provided by the algorithm	Administration of antihistamines, glucocorticoids or epinephrin during the course of re-exposure	Computerized physician order entry

Study population and delabeling outcomes

An overview of the study results is presented in Table 2. During the study period, 30824 patients were hospitalized in UZ Leuven. Of these, 1850 (6.0%) had an unspecified allergy label for penicillins. The Allergy Fact Checker identified 8.9% (164/1850) of these patients as fulfilling all alert criteria and 152 (92.7%, 152/164) were deemed eligible for delabeling after EPR review by the clinical pharmacist. Baseline characteristics are shown in Table 3.

Table 2. Penicillin allergy prevalence and delabeling outcomes in adult inpatients

	Pre-intervention	Post-intervention	Overall
Hospitalized patients, n	17337	16814	30824
Hospitalized patients with penicillin allergy label, n (%)	1055 (6.0)	985 (5.9)	1850 (6.0)
Patients identified by allergy fact checker, n (%)	99 (9.4)	87 (8.8)	164 (8.9)
Patients eligible for delabeling after EPR review, n (%)	93 (94)	79 (91)	152 (93)
Patients delabeled, n	11	47	58
% of hospitalized patients with a penicillin allergy label	1.0	4.8	3.1
% of patients identified by allergy fact checker	11	54	35
% of patients eligible for delabeling after EPR review	12	59	38

EPR electronic patient record.

Table 3. Baseline characteristics of patients eligible for delabeling

	Pre-intervention (93 pts, 136 hospitalizations)	Post-intervention (79 pts, 93 hospitalizations)
Age, mean ± s.d.	67 ± 14	66 ± 15
Sex (male), n (%)	34 (37)	37 (47)
Symptoms of allergic reactiona, n (%)
Anaphylactic shock	2 (2)	0 (0)
Angioedema	6 (6)	6 (8)
Dyspnea	3 (3)	1 (1)
Urticaria	4 (4)	4 (5)
Rash	55 (59)	43 (54)
Itching	4 (4)	5 (6)
Other	25 (27)	23 (29)
Hospitalization ward, n (%)
Pulmonology	30 (22)	19 (20)
Abdominal surgery	11 (8)	4 (4)
Geriatrics	10 (7)	6 (6)
Trauma surgery	10 (7)	5 (5)
Nephrology	8 (6)	3 (3)
Cardiology	7 (5)	1 (1)
Digestive oncology	7 (5)	3 (3)
Gastro-enterology	6 (4)	4 (4)
Hepatology	6 (4)	9 (10)
Neurosurgery	6 (4)	3 (3)
General internal medicine	5 (4)	8 (9)
Hematology	5 (4)	5 (5)
Other	25 (18)	23 (25)
Length of stay, median (IQR)	5 (2–10)	6 (2–12)

Age, sex and symptoms of allergic reaction are reported once per unique patient (based on the first hospitalization). Hospitalization ward and length of stay are reported per hospitalization episode, as patients could be admitted more than once and to different wards.

^aSymptoms of allergic reaction as reported in the allergy label in the electronic patient record. Multiple symptoms could be registered per patient; therefore, the total number of symptoms exceeds the number of patients.

In the pre-intervention period, the Allergy Fact Checker identified 99 unique patients, of whom 93 were deemed eligible for delabeling. These 93 patients accounted for a total of 136 hospitalizations. Actual delabeling occurred in 11 patients (12% of patients, 8% of hospitalizations). Delabeling in these patients was carried out independently by the treating physician, without study intervention.

In the post-intervention period, 87 unique patients were identified by the Allergy Fact Checker, with 79 deemed eligible for delabeling. These 79 patients accounted for a total of 93 hospitalizations. Delabeling recommendations were provided for 83 of the 93 hospitalizations. In the remaining 10 cases, the patient had already been discharged before the EPR review could be completed and none of these patients were delabeled. Of the 83 recommendations, 20 were immediately delabeled by the treating physician. In 23 cases, the recommendation was not (yet) accepted at the time of discharge, and it could not be orally discussed because the patient had been discharged in the meantime. For the remaining 40 cases, the physician was contacted by phone to discuss the recommendation, which led to 26 patients (65% of orally discussed cases) being additionally delabeled. One patient was spontaneously delabeled, without intervention from the Allergy Fact Checker. In total, 47 patients (59% of patients, 51% of hospitalizations) were delabeled during the post-intervention period.

Interrupted time series analysis

The probability of delabeling over time and by period with 95% CIs is presented in Fig. 1. The probability of delabeling was 0.07 (95% CI 0.02–0.23) pre-intervention and significantly increased to 0.32 (95% CI 0.18-0.51) afterwards. An odds ratio (OR) of 6.8 (95% CI 1.3–35) as a result of the implementation of the Allergy Fact Checker was found (P = 0.02). There was no evidence for an underlying time trend pre-intervention (OR 0.93 (95% CI 0.61–1.4)), indicating that the probability of delabeling was consistent over time. However, a significant post-intervention time trend was observed (OR 1.4 (95% CI 1.1–1.8)), indicating that delabeling increased gradually after introduction of the Allergy Fact Checker. The change in trend was not statistically significant (P = 0.12).

Fig. 1 Interrupted time series analysis. [Images not available. See PDF.]

The estimated probability of delabeling with 95% confidence intervals is presented over time and by period. Time is centered at the intervention. The pre-intervention period is presented in red, the post-intervention period is black. (Created with SAS software version 9.4 for Windows).

Interviews with treating physicians

Between 24 and 72 h after sending the electronic note, recommendations were discussed with treating physicians in cases where patients had not yet been delabeled, i.e., in 40 cases. Most common reasons included: (i) alert overlooked due to workload or time constraints (35%); (ii) limited knowledge of the allergy label removal process in the EPR (20%), and (iii) a combination of these two factors (32.5%) (Table 4).

Table 4. Reasons for delayed or non-acceptance of delabeling recommendations (findings from follow-up tele-interviews, n = 40)

Reported reason	n (%)
Alert overlooked due to workload or time constraints	14 (35)
Limited knowledge of the allergy label removal process in the EPR	8 (20)
Combination of alert being overlooked and limited knowledge of the allergy label removal process in the EPR	13 (32.5)
Recommendation deemed irrelevant due to recent initiation of palliative care	2 (5)
Preference to consult a supervising physician before removing the allergy label	2 (5)
Unfamiliarity with penicillin allergy leading to discomfort with removing the label	1 (2.5)

EPR electronic patient record.

Performance of the Allergy Fact Checker in the exploratory analysis

A convenience sample of 100 patients with a penicillin allergy label was selected. The sensitivity was 68% (17/25), specificity 99% (74/75), positive predictive value (PPV) 94% (17/18), and negative predictive value (NPV) 90% (74/82). All eight false-negative cases were due to outpatient re-exposure documented in free-text, which were not captured by the algorithm (Table 5).

Table 5. Performance of the Allergy Fact Checker in the exploratory analysis

		Reference (EPR review)			Predictive value
		Positive (should be delabeled)	Negative (should not be delabeled)	Total
Allergy Fact Checker	Positive (alert)	17 (TP)	1 (FP)	18	PPV 94%
	Negative (no alert)	8 (FN)	74 (TN)	82	NPV 90%
	Total	25	75	100
Sensitivity and specificity		Sensitivity 68%	Specificity 99%

EPR electronic patient record, NPV negative predictive value, PPV positive predictive value.

Positive predictive value of the Allergy Fact Checker in the study population

The PPV was defined as the proportion of alerts generated by the Allergy Fact Checker that, after review of the EPR, led to a recommendation for delabeling. The PPV was 94% (93/99) and 91% (79/87) pre-intervention and post-intervention, respectively. Reasons that alerts did not lead to a recommendation for delabeling included: (i) documentation of a reaction during re-exposure (n = 5); (ii) index reaction on piperacillin (n = 4); (iii) index reaction on cefuroxime (n = 1); (iv) re-exposure occurred at the same moment of allergy registration (and EPR review showed it was actually the index reaction instead of re-exposure) (n = 1); (v) date of index reaction unclear (n = 1), and (vi) culprit drug unclear (n = 1).

Time investment

The mean time spent to review one alert was 5 min (±3 min). An allergy specialist was consulted in only one case.

Discussion

We developed and implemented the Allergy Fact Checker, a novel clinical decision support system designed to identify uneventful re-exposure to penicillins in patients with a reported penicillin allergy, proactively facilitating the removal of incorrect labels. The Allergy Fact Checker led to a significant and sustained increase in delabeling (OR 6.8, P = 0.02), demonstrating its effectiveness. The proportion of delabeled patients increased from 12% to 60%. The main barriers encountered by physicians included workload, time constraints and a lack of knowledge regarding the allergy label removal process in the EPR.

The ITS analysis indicated that during the pre-intervention period, the probability of delabeling was low and remained consistent over time. After introduction of the Allergy Fact Checker, an immediate and substantial increase in delabeling was seen (OR 6.8, 95% CI 1.3–35)). Additionally, delabeling continued to rise gradually over time (OR 1.4, 95% CI 1.1–1.8). Possible explanations for this positive time trend are physicians becoming more familiar with the topic, increased awareness, and knowledge of the allergy label removal process. The change in trend was not statistically significant, which may be attributed to insufficient power. Our analysis was explicitly powered to detect the first intervention effect, while the effects on trends were considered exploratory.

The Allergy Fact Checker is a simple and novel ‘if-then’ clinical rule that can be easily integrated into hospital software, making it widely applicable across various healthcare settings. It complements existing delabeling methods (i.e., a comprehensive allergy history to identify clearly non-allergic patients²³, direct oral challenge for low-risk patients¹⁴ and a comprehensive allergy workup including sequential skin and provocation testing for others²⁴). The Allergy Fact Checker specifically targets cases with documented re-exposure readily available, omitting the need for further oral challenge testing. It does so independent of risk classification, allowing other, more time-consuming strategies to be reserved for cases lacking such data.

A few studies, primarily from the US and Australia, have explored the potential of non-invasive delabeling, with delabeling rates ranging from 13 to 64%^{16,18, 19–20,25,26}. Uneventful re-exposure to the culprit drug was shown to be the reason for delabeling in 30 to 95% of cases^{16, 17–18,20}, which the Allergy Fact Checker is designed to identify efficiently, streamlining the delabeling process.

We opted for a secondary review of the alerts by a specifically trained person, in our case, a clinical pharmacist, before notifying the treating physician. This decision was based on literature indicating that systems directly addressing prescribers are often less effective, due to factors such as workload and alert fatigue^27,28. Clinical pharmacists have dedicated time to review alerts, assess their relevance and collaborate with prescribers when necessary, which enhances the uptake of clinical decision support alerts²².

This was further supported by the interviews, where the vast majority of physicians agreed to remove the allergy label, but identified workload, time constraints and limited knowledge of the allergy label removal process as the main barriers to doing so. Discussing the recommendation with treating physicians reinforced the importance of delabeling and clarified the allergy label removal process, resulting in a substantial increase in delabeled patients. Importantly, delabeling should be a simple and straightforward action in the EPR, and collaboration with EPR vendors is crucial to streamline this process.

The PPV of the Allergy Fact Checker was notably high (92.7%), resulting in very efficient use of resources. The Allergy Fact Checker identified 9% of alleged penicillin-allergic patients, which represents a quick win given the very high PPV and minimal time investment required for review. However, the exploratory performance analysis showed a lower sensitivity and NPV of 68% and 90%, suggesting that more patients could benefit from our intervention. The cohort of potential delabeling candidates could be expanded by integrating natural language processing to extract meaningful information from unstructured data in the EPR which is not currently captured by the Allergy Fact Checker.

The average time to review and address one alert was only 5 minutes, a stark contrast to the time-consuming questionnaires, manual EPR searches, and allergy workups typically required in delabeling strategies^12,16,29, as well as the negative outcomes associated with choosing less effective and broader spectrum antimicrobials when the allergy label is not critically assessed, which is the case in the majority of labels³⁰. Song et al.²⁵ reported an average of 5.2 min per patient questionnaire, excluding the additional time required for chart reviews and the documentation post-interview. Moreover, only 24% could potentially be delabeled following such interviews, compared to 92.7% in our study.

Our study has several important strengths. Firstly, to our knowledge, we are the first to develop and evaluate a clinical decision support tool designed for direct penicillin allergy delabeling. This tool identifies those patients who can be delabeled immediately, bypassing more time-consuming delabeling strategies such as direct oral challenges and allergy workups, and operates independently of risk scores. Secondly, a major strength of our study was the mixed-methods design. By combining quantitative data from the ITS analysis with qualitative insights from the physician’s interviews, we comprehensively evaluated the Allergy Fact Checker and its implementation in clinical practice. Additionally, ITS is acknowledged as the strongest quasi-experimental approach for evaluation of longitudinal effects of interventions. Thirdly, our intervention was launched directly in clinical practice, enhancing the reliability of our results in a real-world setting. Lastly, the time spent on reviewing the alerts was measured, helping to assess the overall impact of the intervention on resource utilization.

Our study also has some limitations. Firstly, we did not assess clinical outcomes such as antibiotic use, in-hospital mortality, length of stay, and ICU admission rates. Our study aimed to assess the potential of the Allergy Fact Checker on delabeling and to identify potential barriers to its implementation in clinical practice. However, we acknowledge that clinical outcomes are the most appropriate metric to measure the actual impact of our intervention, and this should be assessed in future research. Secondly, the Allergy Fact Checker’s potential could be further enhanced by incorporating language models to process free-text information, as suggested by our exploratory analysis. Thirdly, the exploratory analysis estimating the performance metrics of the Allergy Fact Checker was based on a small sample of 100 patients which may limit the generalizability of the findings to the broader population. Therefore, these results should be interpreted with caution and viewed primarily as a means to understand the current impact and reach of the tool and to guide further improvement. Fourthly, the Allergy Fact Checker relies on prescribing events despite existing allergy labels, and its applicability and reach may vary with local prescribing practices. Fifthly, we did not formally monitor the safety of our intervention, i.e., potential allergic reactions following allergy delabeling. However, no incidents were reported during the study period. Finally, we focused solely on delabeling within the hospital setting. Discordant labeling across patient, primary, and secondary care levels may occur due to the absence of a centralized labeling system in Belgium, as is the case in many other countries. This exacerbates the inefficiency of managing allergy labels, as updates made in one setting may not be reflected in others.

In conclusion, the Allergy Fact Checker is a novel and easily implementable clinical decision support system that significantly enhances proactive penicillin allergy delabeling. This approach requires minimal resources, making it easy to integrate into existing clinical workflows.

Methods

Study design

A convergent mixed-methods design was applied, including both a quantitative and a qualitative evaluation of our intervention.

First, a quasi-experimental ITS analysis was performed to evaluate the impact on delabeling. The ITS design is characterized by a series of measurements over time, interrupted by an intervention, i.e., the implementation of the Allergy Fact Checker in the hospital information system. The primary outcome was penicillin allergy delabeling (yes/no) at the hospitalization level. Data collection at multiple time points before and after the implementation of the intervention allowed evaluation of the effect over time (trend) of each period and the abrupt change in level as a result of the intervention (immediate effect of the intervention)³¹.

Second, barriers regarding actual delabeling in the EPR were studied by tele-interviews with treating physicians.

The study was approved by the Ethics Committee Research UZ/KU Leuven (S61615). The study was implemented as a quality improvement project with a waiver of informed consent.

Setting and patients

The study took place in UZ Leuven, a large (1650 inpatient beds) tertiary academic hospital in Belgium. At UZ Leuven, all patients except those admitted to the intensive care unit have an EPR (Clinical Workstation), which includes computerized physician order entry (CPOE) and multiple clinical decision support systems, including an allergy registration module. The allergy module was introduced in 2010 and has been previously described in detail^1,20. Briefly, this module allows healthcare providers to register a specific drug allergy, including the onset date and the type of allergic reaction, along with free text comments. When a drug is prescribed for which an allergy has been registered, automated safety alerts are triggered based on the severity of the index reaction.

The study population comprised all non-critically ill adult inpatients with a penicillin allergy label documented in their EPR, with the possibility of including unique patients multiple times (once per hospitalization).

Performance of the Allergy Fact Checker

An exploratory analysis was conducted to assess the performance of the final version of the Allergy Fact Checker. A convenience sample of 100 patients with a penicillin allergy label was selected. A clinical pharmacist from the research group reviewed each case to determine whether the patient had experienced a re-exposure to the culprit drug after the allergy was registered, and whether this occurred without allergic symptoms. Sensitivity, specificity, PPV, and NPV were calculated using this manual EPR review as the reference standard.

Study procedure and data collection

Interrupted time series analysis

The pre-intervention period took place from 01/11/2023 until 14/04/2024. During this period, the clinical rule alerts were reviewed on weekdays by a clinical pharmacist. The possibility for delabeling was documented in the interest of the study. No recommendation to the treating physician was provided at this point.

The post-intervention period took place from 15/04/2024 until 27/09/2024. During this period, again, clinical rule alerts were reviewed on weekdays by a clinical pharmacist. Additional interpretation by an allergy specialist could be requested when required. When delabeling was deemed possible, a recommendation was provided in the EPR addressing the treating physician, and this was done once per hospital admission. The note included a summary of the algorithm’s findings (i.e., the date of the allergy label and any subsequent re-exposures without allergic symptoms), a request for review and delabeling of the patient (i.e., deleting the allergy label in the EPR), a reference to the in-house delabeling procedure, and a phone number for questions and additional information. The time required to review each alert was registered.

Interviews with treating physicians

Between 24 and 72 h after sending the recommendation, treating physicians were contacted by phone if delabeling had not yet been performed and the patient was still hospitalized. During this phone call, the recommendation was further discussed, and reasons and arguments driving their decision-making, including non-acceptance, were questioned. Notes were taken during these interviews.

Outcomes

The primary outcome was penicillin allergy delabeling (yes/no) at the hospitalization level. The specific moment of delabeling (after the electronic note or the phone call) was documented.

Secondary outcomes included delabeling at the patient level, barriers encountered by physicians in the allergy label removal process, the PPV of the Allergy Fact Checker in the pre- and post-intervention study cohorts and time investment required to review the alerts.

Barriers were assessed through notes from physician interviews. The PPV was defined as the proportion of alerts generated by the Allergy Fact Checker that, after review of the EPR, led to a recommendation for delabeling, i.e., true positives divided by the total number of alerts. Time investment was measured by documenting the start and stop times of alert review, including documentation of the recommendation in the EPR.

Data analysis

Descriptive statistics were presented as frequency with percentage for categorical data and mean with standard deviation for continuous data.

Interrupted time series analysis

Given the binary outcome, the data was analyzed using a logistic regression model. Estimated effects are expressed as OR with 95% confidence intervals (CI)³¹. Generalized estimating equations (GEE) were used to account for data clustering, as individual patients could be included more than once (i.e., during separate hospitalizations)³². GEE adjusted for within-patient correlations, to ensure valid estimation of standard errors and population-averaged effects.

A sample size calculation was performed based on an expected delabeling rate of 5% pre-intervention and 50% post-intervention. To achieve a power of 80%, 84 patients were required in each period.

The statistical analyses were carried out with SAS software version 9.4 for Windows.

Interviews with treating physicians

Textual qualitative data gathered by telephone contact with physicians were analyzed using inductive thematic analysis. Answers were grouped and recoded independently by a clinical pharmacist of the research group and a senior clinical pharmacist until consensus was reached for the interpretation of all answers.

Acknowledgements

The authors would like to thank Valerie Caubergs, Matthias Gijsen, Charlotte Quintens, Lotte Vander Elst and Evelyne Van den Broucke for reviewing the alerts of the Allergy Fact Checker during GVDS’s holiday absence. L.G. is funded by a Fonds Wetenschappelijk Onderzoek - Flanders National Fund for Scientific Research (FWO) junior postdoctoral fellowship (12A6Q24N). R.S. is funded by a FWO senior clinical investigator fellowship (1805518N). I.S. is funded by the Clinical Research Fund, UZ Leuven. This work was supported by Applied Biomedical Research grant of the Flemish government (FWOTBM), Belgium (T003023N). Neither the funder nor any external party had a role in the data collection, analysis interpretation, writing of the manuscript, or the decision to submit for publication.

Author contributions

G.V.D.S., L.G., D.W., C.B., T.I., P.D.M., R.S., and I.S. conceptualized and designed the study. G.V.D.S. developed the content of the Allergy Fact Checker, collected and analyzed the data and wrote the manuscript. L.G. verified the data. G.V.D.S. and R.S. addressed the alerts generated by the Allergy Fact Checker. RS and IS supervised the study. D.W., C.B., T.I., P.D.M., R.S., and I.S. critically reviewed the manuscript. G.V.D.S. and L.G. revised the manuscript. All authors had full access to all the data in the study, have read and approved the final version of the manuscript and agreed to be accountable for their contributions.

Data availability

The data of this study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors used OpenAI ChatGPT in order to improve writing style and check grammar and spelling. After using this tool, the authors reviewed and edited the content as needed and takes full responsibility for the content of the publication.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41746-025-02030-1.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Gilissen, L; Spriet, I; Gilis, K; Peetermans, WE; Schrijvers, R. Prevalence of antibiotic allergy labels in a tertiary referral center in Belgium. J. Allergy Clin. Immunol. Pract.; 2021; 9, pp. 2415-2425.e2418.1:CAS:528:DC%2BB38XitVaqurjL [DOI: https://dx.doi.org/10.1016/j.jaip.2021.01.047] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33607341]

2. Blumenthal, KG; Peter, JG; Trubiano, JA; Phillips, EJ. Antibiotic allergy. Lancet; 2019; 393, pp. 183-198. [DOI: https://dx.doi.org/10.1016/s0140-6736(18)32218-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30558872]

3. Macy, E; Contreras, R. Health care use and serious infection prevalence associated with penicillin “allergy” in hospitalized patients: a cohort study. J. Allergy Clin. Immunol.; 2014; 133, pp. 790-796. [DOI: https://dx.doi.org/10.1016/j.jaci.2013.09.021] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24188976]

4. Charneski, L; Deshpande, G; Smith, SW. Impact of an antimicrobial allergy label in the medical record on clinical outcomes in hospitalized patients. Pharmacotherapy; 2011; 31, pp. 742-747. [DOI: https://dx.doi.org/10.1592/phco.31.8.742] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21923600]

5. Blumenthal, KG; Lu, N; Zhang, Y; Walensky, RP; Choi, HK. Recorded penicillin allergy and risk of mortality: a population-based matched cohort study. J. Gen. Intern Med; 2019; 34, pp. 1685-1687. [DOI: https://dx.doi.org/10.1007/s11606-019-04991-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31011962][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6712108]

6. Wu, JH et al. Potential negative effects of antimicrobial allergy labelling on patient care: a systematic review. Can. J. Hosp. Pharm.; 2018; 71, pp. 29-35. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29531395][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5842048]

7. Kaminsky, LW; Ghahramani, A; Hussein, R; Al-Shaikhly, T. Penicillin allergy label is associated with worse clinical outcomes in bacterial pneumonia. J. Allergy Clin. Immunol. Pr.; 2022; 10, pp. 3262-3269.1:CAS:528:DC%2BB38Xis1equ7rJ [DOI: https://dx.doi.org/10.1016/j.jaip.2022.08.027]

8. Powell, N; West, R; Sandoe, J. Impact of penicillin allergy records on carbapenem prescribing: an observational retrospective cohort study. J. Hosp. Infect.; 2019; 101, pp. 467-470.1:STN:280:DC%2BB3cngtVSitw%3D%3D [DOI: https://dx.doi.org/10.1016/j.jhin.2018.11.020] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30529507]

9. Barlam, TF et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin. Infect. Dis.; 2016; 62, pp. e51-e77. [DOI: https://dx.doi.org/10.1093/cid/ciw118] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27080992][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5006285]

10. Savic, L et al. BSACI guideline for the set-up of penicillin allergy de-labelling services by non-allergists working in a hospital setting. Clin. Exp. Allergy; 2022; 52, pp. 1135-1141. [DOI: https://dx.doi.org/10.1111/cea.14217] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36128691]

11. Wijnakker, R et al. The Dutch Working Party on Antibiotic Policy (SWAB) guideline for the approach to suspected antibiotic allergy. Clin. Microbiol Infect.; 2023; 29, pp. 863-875.1:CAS:528:DC%2BB3sXptVOltL8%3D [DOI: https://dx.doi.org/10.1016/j.cmi.2023.04.008] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37068548]

12. Khan, DA et al. Drug allergy: a 2022 practice parameter update. J. Allergy Clin. Immunol.; 2022; 150, pp. 1333-1393. [DOI: https://dx.doi.org/10.1016/j.jaci.2022.08.028] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36122788]

13. Cooper, L; Harbour, J; Sneddon, J; Seaton, RA. Safety and efficacy of de-labelling penicillin allergy in adults using direct oral challenge: a systematic review. JAC Antimicrob. Resist; 2021; 3, [DOI: https://dx.doi.org/10.1093/jacamr/dlaa123] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34223072][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8210118]dlaa123.

14. Copaescu, AM et al. Efficacy of a clinical decision rule to enable direct oral challenge in patients with low-risk penicillin allergy: the PALACE randomized clinical trial. JAMA Intern Med.; 2023; 183, pp. 944-952.1:CAS:528:DC%2BB3sXitValsLvF [DOI: https://dx.doi.org/10.1001/jamainternmed.2023.2986] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37459086][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10352926]

15. Trubiano, JA et al. Development and validation of a penicillin allergy clinical decision rule. JAMA Intern Med.; 2020; 180, pp. 745-752. [DOI: https://dx.doi.org/10.1001/jamainternmed.2020.0403] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32176248]

16. Chua, KYL et al. The penicillin allergy delabeling program: a multicenter whole-of-hospital health services intervention and comparative effectiveness study. Clin. Infect. Dis.; 2021; 73, pp. 487-496.1:CAS:528:DC%2BB3MXhslyjsrjI [DOI: https://dx.doi.org/10.1093/cid/ciaa653] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32756983]

17. du Plessis, T; Walls, G; Jordan, A; Holland, DJ. Implementation of a pharmacist-led penicillin allergy de-labelling service in a public hospital. J. Antimicrob. Chemother.; 2019; 74, pp. 1438-1446.1:CAS:528:DC%2BC1MXitlantr7N [DOI: https://dx.doi.org/10.1093/jac/dky575] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30753497]

18. Devchand, M et al. Evaluation of a pharmacist-led penicillin allergy de-labelling ward round: a novel antimicrobial stewardship intervention. J. Antimicrob. Chemother.; 2019; 74, pp. 1725-1730.1:CAS:528:DC%2BB3cXktVyhuro%3D [DOI: https://dx.doi.org/10.1093/jac/dkz082] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30869124]

19. Torda, A; Chan, V. Antibiotic allergy labels-the impact of taking a clinical history. Int. J. Clin. Pract.; 2018; 72, e13058.1:CAS:528:DC%2BC1cXot1eqt74%3D [DOI: https://dx.doi.org/10.1111/ijcp.13058] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29341373]

20. Van De Sijpe, G et al. Non-invasive delabeling and refining of beta-lactam allergy labels in inpatients to optimize antimicrobial stewardship. Allergy; 2022; 77, pp. 3157-3159. [DOI: https://dx.doi.org/10.1111/all.15453] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35876226]

21. Romano, A; Gaeta, F; Arribas Poves, MF; Valluzzi, RL. Cross-reactivity among beta-lactams. Curr. Allergy Asthma Rep.; 2016; 16, 1:CAS:528:DC%2BC28Xjt1Onur8%3D [DOI: https://dx.doi.org/10.1007/s11882-016-0594-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26898316]24.

22. Quintens, C., Van de Sijpe, G., Van der Linden, L. & Spriet, I. Computerised prescribing support still needs a human touch. Age Ageinghttps://doi.org/10.1093/ageing/afaa200 (2020).

23. Stone, CA, Jr; Trubiano, J; Coleman, DT; Rukasin, CRF; Phillips, EJ. The challenge of de-labeling penicillin allergy. Allergy; 2020; 75, pp. 273-288.1:STN:280:DC%2BB3MbotVWntg%3D%3D [DOI: https://dx.doi.org/10.1111/all.13848] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31049971]

24. Mirakian, R et al. Management of allergy to penicillins and other beta-lactams. Clin. Exp. Allergy; 2015; 45, pp. 300-327.1:CAS:528:DC%2BC2MXktFarur4%3D [DOI: https://dx.doi.org/10.1111/cea.12468] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25623506]

25. Song, YC; Nelson, ZJ; Wankum, MA; Gens, KD. Effectiveness and feasibility of pharmacist-driven penicillin allergy de-labeling pilot program without skin testing or oral challenges. Pharmacy (Basel); 2021; 9, 127. [DOI: https://dx.doi.org/10.3390/pharmacy9030127] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34287342]

26. Sigona, NS; Steele, JM; Miller, CD. Impact of a pharmacist-driven beta-lactam allergy interview on inpatient antimicrobial therapy: a pilot project. J. Am. Pharm. Assoc. (2003); 2016; 56, pp. 665-669. [DOI: https://dx.doi.org/10.1016/j.japh.2016.05.005] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27745795]

27. Blanc, AL et al. Prevention of potentially inappropriate medication in internal medicine patients: a prospective study using the electronic application PIM-Check. J. Clin. Pharm. Ther.; 2018; 43, pp. 860-866. [DOI: https://dx.doi.org/10.1111/jcpt.12733] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29978537]

28. O’Mahony, D et al. Prevention of adverse drug reactions in hospitalized older patients with multi-morbidity and polypharmacy: the SENATOR* randomized controlled clinical trial. Age Ageing; 2020; 49, pp. 605-614. [DOI: https://dx.doi.org/10.1093/ageing/afaa072] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32484850]

29. Romano, A et al. Towards a more precise diagnosis of hypersensitivity to beta-lactams - an EAACI position paper. Allergy; 2020; 75, pp. 1300-1315. [DOI: https://dx.doi.org/10.1111/all.14122] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31749148]

30. Samarakoon, U et al. Penicillin allergy delabeling: opportunities for implementation and dissemination. Ann. Allergy Asthma Immunol.; 2023; 130, pp. 554-564. [DOI: https://dx.doi.org/10.1016/j.anai.2022.12.023] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36563744]

31. Wagner, AK; Soumerai, SB; Zhang, F; Ross-Degnan, D. Segmented regression analysis of interrupted time series studies in medication use research. J. Clin. Pharm. Ther.; 2002; 27, pp. 299-309.1:STN:280:DC%2BD38vhvFOnuw%3D%3D [DOI: https://dx.doi.org/10.1046/j.1365-2710.2002.00430.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12174032]

32. Hanley, JA; Negassa, A; Edwardes, MD; Forrester, JE. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am. J. Epidemiol.; 2003; 157, pp. 364-375. [DOI: https://dx.doi.org/10.1093/aje/kwf215] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12578807]