Behavioural economics to improve and motivate vaccination in primary care using nudges through the electronic health record: rationale and design of the BE IMMUNE randomised clinical trial

Abstract

Introduction

Annual influenza vaccination reduces disease burden but vaccination rates are suboptimal, with persistent disparities among subpopulations. The purpose of this trial is to evaluate multicomponent behavioural economic nudge interventions to clinicians and patients to increase influenza vaccination. This trial also includes an intensification nudge to reduce disparities in vaccination among older adult, primary care patients.

Methods

This is a two-part, multisite cluster randomised, pragmatic clinical trial. In the first part, a multicomponent nudge intervention will be tested over approximately 6 months (September 2023–February 2024). The second part consists of a replication trial conducted at an additional site during the following influenza season (September 2024–February 2025). Primary care clinics will be randomised to the nudge intervention or usual care. Eligible clinicians and patients at intervention clinics will receive the intervention, and patients deemed high risk for not receiving a vaccine will be further randomised to receive an intensification nudge. The primary outcome is vaccine completion during the eligible visit and the secondary outcome is vaccine completion within 3 months of the eligible visit.

Analysis

The effect of the clinic-level nudge intervention on the primary and secondary outcomes will be evaluated using generalised estimating equations (GEEs) with a clinic-level exchangeable working correlation to account for clustering of observations within the clinic. GEE models with an independent working correlation will be used to evaluate the impact of the additional intensification nudge on the primary and secondary outcomes.

Ethics and dissemination

The University of Pennsylvania Institutional Review Board (IRB) approved this trial and serves as the single IRB of record (IRB #851838). Results will be disseminated via peer-reviewed publication and conference presentations.

Trial registration number

NCT06057727.

Correspondence to Dr Kimberly Waddell; [email protected]

STRENGTHS AND LIMITATIONS OF THIS STUDY

• Multicomponent, personalised behavioural economic nudge interventions targeted to both primary care clinicians and patients at three distinct sites.

• Includes an additional, intensification nudge for patients who are at high-risk for vaccine non-participation to reduce persistent disparities in influenza vaccination.

• Leverages automation to place a default pended influenza vaccine order prior to the eligible patient visit, significantly reducing clinician burden.

• Limitations include findings may not generalise to settings that do not use an electronic health record with functionality for pended orders or patient messaging.

Introduction

Annual vaccination protects individuals from infection, hospitalisation and death from the influenza virus.¹ Approximately 90% of influenza-related deaths occur in adults ≥65 years of age. Despite the protective benefits of the influenza vaccine, completion rates remain suboptimal, with an overall vaccination rate of 46.9% among adults in the USA.¹ Only 50.1% of adults 50–64 years of age and 69.7% of adults ≥65 years of age received the influenza vaccination in the 2022–2023 influenza season.¹ There are patient subgroups who experience additional barriers and/or disparities in care that place them at higher risk for non-participation. For example, the influenza vaccination rates for individuals who are racial/ethnic minorities range from 36% to 44% in the USA.¹

In recent years, there has been a shift towards leveraging digital technologies as cost-efficient, scalable solutions for addressing persistently suboptimal vaccination rates. With over 90% of health systems using an electronic health record (EHR), there are opportunities to implement EHR-based interventions to promote increased vaccination.^2–4 Text messaging has emerged as another low-cost, scalable technology for engaging patients in their healthcare. Unfortunately, while they possess certain advantages over analogue, mail or phone-based strategies, there is limited evidence on the effectiveness of EHR and text message-based interventions to reach national vaccination targets.^5–13

One notable limitation of these prior digital interventions was not accounting for known decision biases, informed by the field of behavioural science, which can have an outsized impact on decisions by clinicians to order vaccines or decisions by patients to receive them. For instance, individuals can be biased towards the status quo, or the choice that occurs if no action is taken.^{14 15} Unfortunately, in most practice settings, the default choice is not to order or receive the influenza vaccine. Clinicians and patients must actively choose to order and receive vaccines, respectively. Additionally, the way information is framed or workflow processes also impact behaviour.¹⁶ Traditionally, standard communication about the importance of receiving an influenza vaccine includes general education without consideration for how to provide information in a way that may increase motivation to act, or make it easier to complete the vaccine.

One way to target decision biases is via ‘nudges’, interventions designed using behavioural science principles to improve health decisions and behaviours.^{2 17} Indeed, single-component nudge interventions can be implemented within the EHR or via text messages to improve preventive care.^18–20 Prior studies of nudges—such as defaults, endowment framing and peer comparison—increased vaccination rates for influenza and the COVID-19 virus.^21–24 These prior nudges were single-component and directed to either a clinician or patient, but not both. Given the persistently suboptimal national influenza vaccination rates, multicomponent, multirecipient nudge interventions—that is, those that simultaneously address multiple decision biases among both clinicians and patients—may be needed to increase the impact of nudge interventions and promote influenza vaccine towards national goals. Additionally, prior nudge interventions have not directly addressed the persistent disparities among subpopulations. Designing targeted nudges for subpopulations who experience persistent disparities in influenza vaccination may help to reduce these disparities, but this strategy has not been tested in prior work.

Currently, there is a knowledge gap in how to design multicomponent nudges and implement them simultaneously among both clinicians and patients to promote preventive health behaviours such as influenza vaccination. There is also a dearth of data about how to tailor such multicomponent nudges to address disparities among individuals at high risk for not receiving vaccines. To address this knowledge gap, this two-part clinical trial is designed to systematically test a multicomponent digital nudge intervention among clinicians and older adults in primary care settings across three health systems, with an additional tailored, intensification nudge for patients at higher risk for vaccine non-participation.

Methods

The Behavioural Economics to Improve and Motivate Vaccination Using Nudges through the EHR (BE IMMUNE) study is a two-part, multisite cluster randomised, pragmatic clinical trial to increase influenza vaccine completion using personalised nudges to clinicians and patients (NCT06057727). The trial was approved by the University of Pennsylvania Institutional Review Board, which served as the single IRB of record and a waiver of participant informed consent was granted. Reporting of this protocol follows the Standard Protocol Items: Recommendations for Interventional Trials guidelines.

The three aims of the BE IMMUNE trial are to (1) examine the effectiveness of personalised nudges to clinicians and patients to increase influenza vaccination among older adults; (2) evaluate the effectiveness of an additional ‘intensification’ nudge to reduce disparities in influenza vaccination among patient subgroups at high risk for vaccine non-participation and (3) assess heterogeneity in treatment effect across clinicians, patients and clinic.

Patient and public involvement

Patients and clinicians were engaged during the design phase of this trial. Primary care patients (n=3) reviewed and provided feedback on all text message content. Patients were also asked about the content of the bidirectional high-risk intensification nudge and had the opportunity to suggest additional topics that were included in the menu. Additionally, we visited primary care clinicians (n=2) and one advanced practice provider to observe their clinical workflows and discuss the pended default vaccination order. The design and outcomes of the study were also presented to primary care leadership to obtain their feedback on the recruitment plan, study timeline and design of the multicomponent nudge intervention.

Setting and duration

The first part of the BE IMMUNE trial is being conducted over approximately 6 months (September 2023–February 2024) during influenza season in primary care clinics within the University of Pennsylvania (Penn) and University of Washington (UW) health systems. In the second part, a replication trial is planned at a third health system site, Lancaster General Health in Lancaster, Pennsylvania, for approximately 6 months during the 2024–2025 influenza season.

Eligibility and screening

This trial includes clinician and patient eligibility criteria. A total of 48 primary care clinics across both Penn and UW were included and randomised. Within intervention clinics, clinicians are eligible to receive the peer comparison nudge (detailed below) if their role is physician or advanced practice provider with a minimum panel of 50 patients. Resident and fellow physicians were not included in the peer comparison nudge. All clinicians who primarily practice at an intervention clinic are eligible to receive the automated pended order nudge (detailed below), regardless of panel size. Resident and fellow physicians were included in the automated pended order nudge. Patients are eligible for inclusion if they are ≥50 years old, eligible to receive an influenza vaccine and have not yet received one during the applicable influenza season and are scheduled for a new or non-urgent/sick return primary care visit at an intervention clinic. Patients are excluded from the trial if they have a documented allergy to the influenza vaccine or an influenza vaccine exclusion modifier within their medical chart. The exclusion modifier is placed for at least one of the following reasons: clinician-initiated exclusion, permanent contraindication for the influenza vaccine or patient permanently refused. Patients who have previously opted out of being contacted for research studies per individual health system guidelines or those without a listed mobile or home telephone number are also excluded.

Based on analysis of Penn and UW health system EHR data prior to the trial, four criteria were identified as being associated with not receiving an influenza vaccination: ≥70 years of age; residence in a low-income neighbourhood (lowest quartile of neighbourhood-level median household income); non-completion of influenza vaccination in the prior year; self-identification of non-Hispanic black race. Individuals meeting one or more of these criteria are eligible for an additional tailored ‘intensification’ nudge.

All patients are screened for eligibility and enrolled 4 days prior to their eligible primary care visit using daily queries of Clarity, Epic’s reporting database. Queries are integrated with Way to Health, a research technology platform used in prior trials to support patient enrolment, randomisation and automated text messaging.²⁵

Randomisation

This trial includes two levels of randomisation (figure 1). First, primary care clinics are randomised using a 2:1 ratio to either the intervention arm or usual care arm using covariate-constrained randomisation.²⁶ Covariate-constrained randomisation first enumerates a large number (eg, 50 000) of possible intervention allocations. Then, a measure of covariate balance is computed with respect to a set of prespecified covariates for each possible allocation. Finally, from a subset of possible allocations that achieve adequate covariate balance, one is randomly chosen as the final allocation of interventions for the study. This trial implements a version of covariate-constrained randomisation that is stratified by study site (Penn and UW) and aims to balance the distribution of clinic size and panel risk across trial arms. The randomisation procedure is implemented in R²⁷ V.4.2 using the cvcrand package.²⁸

Figure 1. BE IMMUNE trial overview. BE IMMUNE, Behavioural Economics to Improve and Motivate Vaccination Using Nudges through the electronic health record (EHR).

All patients with eligible visits at a clinic randomised to the intervention arm will receive the multicomponent nudge intervention. Patients included in the clinic-level intervention arm and identified as high risk for vaccine non-participation are further randomised using a 1:1 ratio to receive vs not receive an additional intensification nudge (figure 1). Individual-level randomisation of high-risk patients is stratified by intervention clinic and implemented in Way to Health using permuted block randomisation with random block sizes of 2, 4 and 6.

Multicomponent nudge intervention

Clinicians practising in clinics randomised to the usual care arm will receive no nudge interventions. Usual care includes a Health Maintenance topic, visible within the patient’s chart, informing the clinician that the patient is due for an influenza vaccine. Health maintenance is an EHR function that automatically identifies medical care gaps to help keep patients up to date on their chronic or preventive medical care. Patients receiving care at clinics randomised to the usual care arm will receive standard messaging, sent by the health systems, reminding them of the benefits of receiving an influenza vaccine.

The intervention arm consists of a multicomponent nudge intervention targeting both clinicians and patients (figure 2). Overall, the interventions are designed to address multiple behavioural biases with demonstrated relevance to vaccination decisions. These include framing bias, present-time bias, status quo bias, regret avoidance and peer perceptions/social norms (table 1).^{14–16 29}

Figure 2. Timeline of the multicomponent nudge intervention.

Table 1

Overview of behavioural biases and addressing nudge interventions used in the BE IMMUNE trial

Bias	Description	Intervention	Recipient
Framing bias	The way information is framed can impact decision-making	Previsit messages with endowment framing	All patients
Present-time bias	Positive reinforcement can create a preference for shorter-term rewards. These rewards can be reinforced by measures that address uncertainty in logistics, processes and availability	Previsit messages with endowment framing	All patients
Availability bias	Individuals often rely on immediate examples, or what comes to mind, when evaluating a decision	Previsit bidirectional text message	Patients at high risk for vaccine non-participation
Status quo bias	Individuals often prefer the option that occurs if no action is taken	Automated pended order	Clinicians
Regret avoidance	Individuals prefer to avoid feelings of regret, which could be experienced by cancelling a pended order for a vaccine	Automated pended order	Clinicians
Peer perceptions/social norms	Individual behaviour is influenced by social preferences, norms and perceived peer behaviour	Monthly peer comparison feedback	Clinicians

BE IMMUNE, Behavioural Economics to Improve and Motivate Vaccination Using Nudges through the EHR; EHR, electronic health record.

The first clinician nudge consists of a default order for influenza vaccine that is pended prior to a patient’s visit, in preparation for the clinician to sign or cancel during the patient visit. A pended order, in contrast to a standing order, is not active until it is reviewed and signed by a clinician. At the Penn study site, the influenza vaccine order is pended using robotic process automation (RPA), a technology that emulates human interactions with computers to accomplish repetitive tasks for which other forms of automation are unavailable. RPA simulates keyboard and mouse clicks to interact with buttons and inputs in an application. In BE IMMUNE, pended orders are placed in the EPIC EHR system using the Microsoft Power Automate RPA product. At UW, the influenza vaccine order is pended using a customised version of Epic’s PlaceOrdersUsingTemplate Application Programming Interface (API). The default version of this API does not place orders in the clinic-administered medication (CAM) order mode. The API was modified to place CAM orders because it is the standard order mode for outpatient vaccines used by UW. Both the RPA and API workflows use Epic Clarity queries to identify eligible patients in order to place pended orders (standard vaccine for patients age 50–64 years; high-dose vaccine for patients 65 years or older).

The second clinician nudge is peer comparison feedback. This feedback is sent to clinicians in the intervention arm via email on a monthly basis, describing vaccination rates among their eligible patients compared with rates among their peer clinicians’ patients. In the first month of the trial, we used influenza vaccination rates from the prior year to calculate peer comparison data. In subsequent months, a cumulative influenza vaccination rate from the current season was used to calculate peer comparison data. Consistent with prior work,^{30 31} clinicians whose vaccination rate is below the group median are provided feedback relative to the median. Clinicians whose vaccination rate is above the median but below the 90th percentile receive feedback comparing their rate to the 90th percentile. Clinicians with vaccination rates above the 90th percentile receive feedback that they are a high performer, a strategy to counteract regression to mean among top performers.²⁹

The patient nudge intervention consists of a text message 3 days prior to their eligible visit informing them that an influenza vaccine is ‘available’ for them at that upcoming visit. A second text message is sent 24 hours prior to the visit, informing patients that a vaccine has been ‘reserved’ for them.²² Patients who do not have a mobile number listed in the EHR receive an automated voice recording telephone call at 3 days and 24 hours prior to the eligible visit, with identical content as the text messages. Translated text messaging and automated voice recordings are available in Spanish, Simplified Chinese, Vietnamese and Russian. Translated messages are sent to patients based on their language preference in the EHR. The patient messaging is sent via Way to Health based on the daily Epic Clarity data queries. If a patient cancels or reschedules their office visit, messages are not sent and the patient is not enrolled again if another visit is scheduled. Patients can also reply ‘DONE’ if they have received their influenza vaccination and subsequent reminder text messages are stopped. In this circumstance, any pended influenza vaccination order is subsequently cancelled by a member of the study team prior to the patient’s office visit. Lastly, patients can opt-out out of receiving study text messages at any point during the trial.

In the intervention arm, the additional intensification nudge is a bidirectional text message sent to eligible patients (individuals at high risk for vaccine non-participation and randomised to receive to additional intensification nudge) 3 days prior to their eligible visit (figure 2). The bidirectional text provides patients with a menu of options related to barriers or concerns about receiving an influenza vaccine. Patients can respond to the bidirectional text and receive more information about the influenza vaccine relevant to their specific concern(s). For example, if a patient responds indicating concerns with vaccine side effects, a text is returned providing the patient with additional information from the US Centers for Disease Control and Prevention about vaccine side effects. Finally, if the automated responses are unable to sufficiently address the patient’s concern, patients are sent an additional text message 15 min prior to their scheduled appointment that encourages them to discuss their question with their clinician.

Outcomes

The primary outcome is patient influenza vaccination completion during the first eligible primary care visit and the secondary outcome is influenza vaccination completion within 3 months of the eligible primary care visit. The coprimary endpoints for the primary and secondary outcomes are the difference in the proportion of vaccinations completed during the first eligible primary care visit between usual care and clinic-level intervention arm and the difference in vaccine completion proportions between high-risk patients who received versus did not receive the intensification nudge.

Sample size and power

The trial power calculations used an adjusted type I error of 0.025 to account for the two coprimary endpoint comparisons: (1) clinic-level nudges versus usual care and (2) intensification for high-risk patients versus no intensification within practices where clinic-level nudges are provided. Based on health systems data analysed prior to the trial, we have 85% power to detect a 7 percentage point difference in the influenza vaccine completion rate for comparison #1. This calculation assumed N=48 participating clinics across the two sites, an average cluster size of n=1385 patients and an intracluster correlation of 0.02. For comparison #2, there is at least 99% power to detect a 2 percentage point difference even if the rate of vaccination in the high-risk group is 50%, the most conservative rate with respect to power. This calculation assumed 1:1 individual-level randomisation of n=1000 high-risk patients within each of N=32 clinics that will be randomised to receive clinic-level nudges. These power calculations combined retrospective data from Penn and UW to estimate the average number of patients per clinic and the number of high-risk patients who will be randomised.

Statistical analyses

Primary analyses

Primary analyses will assess two comparisons: (1) the impact of the multicomponent nudge intervention (text message to patients; pended orders and peer comparison feedback to clinicians) versus usual care and (2) the impact of the additional intensification nudge (bidirectional text addressing patient concerns) versus usual care. Following the intention-to-treat framework, patients who are randomised and scheduled for—but do not complete—their first eligible visit will be counted as zero for both endpoints (no vaccination at the eligible visit and no vaccination within 3 months of the eligible visit).

Generalised estimating equation (GEE) models will be used for all analyses. To account for the clinic-level covariate-constrained randomisation, models evaluating the impact of clinic-level nudges will include health system, clinic size and panel risk as covariates. An exchangeable working correlation will be specified at the clinic level for the evaluation of clinic-level nudges (comparison #1) to account for clustering of observations within the clinic. To account for time-invariant differences between clinics and the stratification of the individual-level randomisation by the clinic, a fixed effect for each clinic (with the exception of one reference clinic) will be included in all models estimating the effect of intensification among high-risk patients. An independent working correlation will be used for the evaluation of individual-level nudges (comparison #2), reflecting the fact that individual patients were randomised. GEE models are robust to misspecification of the working correlation structure in large samples, this model specification will provide valid statistical inference for the effects of interest even if there is correlation among patients from the same clinic not captured by the clinic fixed effects.

All investigators, statisticians and analysts will be blinded during the intervention and analysis phases. Unblinding will occur at the end of study analysis once all outcomes have been ascertained.

Analyses of vaccine orders

The effects of the interventions on overall rates of clinician vaccine ordering will also be examined. The clinician vaccine ordering rates will be compared between the three intervention arms: (1) control, (2) clinic-level nudges only and (3) clinic-level nudges with intensification for high-risk patients. A GEE will be fit with binary outcome indicating whether the vaccine order was signed or cancelled by the clinician. We will fit the GEE model using the weighted and replicated approach¹³ to account for low-risk patients seen in the intervention clinics. The GEE model will include terms for the main effects of clinic-level and individual-level nudges as well as their interaction. The appropriate linear combinations of the parameters that correspond to pairwise differences between each of the three intervention arms will also be tested. An independence working correlation will be used to account for uncertainty in the weight distribution due to variability in the empirical proportion of high-risk versus low-risk patients.

Sensitivity and exploratory analyses

For exploratory analyses, patients will be stratified into low-risk (patients not at high risk for vaccine non-participation) versus high-risk (patients at high-risk for vaccine non-participation) groups. The primary and secondary endpoint analyses will be repeated for the effects of clinic-level nudges among patients at low risk for vaccine non-participation. Among high-risk patients, we will repeat the primary and secondary endpoint analyses for the effects of individual-level nudges including variables for each of the four criteria for deeming patients high risk for vaccine non-participation (patients over 70 years old; patients residing in low-income neighbourhoods; patients who did not receive an influenza vaccination in the prior year or patients self-identified as non-Hispanic black) and the interaction between these variables with the treatment arm variable. This approach will allow the assessment of which high-risk subgroups benefited the most from the individual-level nudges.

Sensitivity analyses will explore alternative specifications for the working correlation structure to examine robustness of parameter estimates and inference, including a visit-level analysis that clusters at the patient level. Additional sensitivity analyses will also censor patients who have a first eligible primary care visit late in the intervention period (eg, visit in February prior to the end of influenza season in March) and are, therefore, unlikely to receive an influenza vaccine at that visit or have a full 3 months to assess the secondary outcome. Completion rates among the censored patients will also be evaluated.

Discussion

The BE IMMUNE trial represents several key innovations in the effort to improve vaccination. First, it incorporates multiple evidence-based strategies into a multicomponent nudge intervention directed to both clinicians and patients. By combining nudges to both clinicians and patients, BE IMMUNE goes beyond prior work and better accounts for the interplay between clinician and patient decision-making related to influenza vaccination. The trial also has an explicit focus on disparity reduction, incorporating a high-risk intensification nudge that is informed by prior work examining barriers to vaccination among high-risk populations and targeted to patient subpopulations empirically identified as least likely to receive an influenza vaccine.^32–35 This strategy—which seeks to offer patients the opportunity to address salient concerns and encourage early engagement in vaccination decisions—has not been previously tested. Also, this trial demonstrates how interventions can be tailored for specific group through a ‘precision nudge’ approach that does not apply the same intervention to all individuals.

Second, BE IMMUNE will advance knowledge about the role of digital interventions as scalable interventions to address vaccination decisions. The trial leverages automation to pend influenza vaccination orders, a default measure that can significantly reduce clinician cognitive and administrative burden and aligns with growing interest in how automation can improve the efficiency of healthcare delivery.³⁶ More broadly, all components of the intervention are scalable and resource-efficient by virtue of digital format and use of existing technologies (EHR, text messaging). Presently, the EHR at both Penn and UW provides tools for bulk orders to manage population health, but these tools are not available for vaccination orders. The BE IMMUNE trial represents an innovation in the use of automation to provide individualised care at scale, using approaches that are not supported by current bulk ordering functionality. Together, these features reflect a novel approach that moves beyond other nudges, such as active choice interventions,³⁷ that while effective, can induce alert fatigue among clinicians and clinical teams.

Third, the pragmatic design of BE IMMUNE offers a flexible approach to trial implementation that accounts for the uniqueness of different healthcare systems and generates salient, real-world evidence. No two health systems are the same, so interventions within one may need to be adapted to accommodate the circumstances and needs of another. For example, Penn initially used the Epic PlaceOrdersUsingTemplate API, which was used in a prior study to increase liver cancer screening. Based on feedback from the primary care team, the PlaceOrdersUsingTemplate API did not include the associated diagnosis code and a Vaccine Information Statement, two components that were critical for existing workflows. As a result, Penn had to transition to RPA to align with primary care workflow. This was not a workflow requirement at UW, which enabled their ability to use the PlaceOrdersUsingTemplate API for the trial. As a result, digital automation was implemented differently at different study sites—via RPA at one and an API at another—to accomplish the same goal. This pragmatic design provides a blueprint for the BE IMMUNE replication trial, where implementation will need to adapt to the unique needs of an additional primary care setting. Lastly, BE IMMUNE accounts for the fact that usual care can vary by health systems and did not seek to modify usual care practices at both study sites as part of this trial.

Fourth, the design of the BE IMMUNE trial includes a visit-based approach, in contrast to traditional population health approaches that target individuals with mass outreach efforts.^{8 10 11} The pragmatic, visit-based approach is well suited for multicomponent nudge interventions that leverage the EHR. Indeed, in this trial, the timing of the nudge interventions is linked to the primary care visit. The visit-based design affords the opportunity to deliver multicomponent interventions in a timely manner.

Conclusions

The BE IMMUNE clinical trial delivers multicomponent nudge interventions to both primary care clinicians and patients to increase influenza vaccination. The intensification nudge provides a distinct opportunity for those who are at higher risk for vaccine non-participation to engage, in real time, with possible concerns or questions and receive additional information. Although this trial targets influenza vaccination, it establishes a blueprint that, if successful, can be translated to other recommended adult vaccines such as shingles, respiratory syncytial or pneumococcal. Additionally, the implementation of this work across three health systems will advance the field’s understanding of how to leverage EHR-based nudges within different health systems that can be applied in future work.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Contributors KWa: conceptualisation, methodology, writing (original draft, editing), visualisation; SJM, AN, JML: conceptualisation, methodology, reviewing and editing, supervision, funding acquisition; KL: conceptualisation, methodology, analysis, reviewing and editing; S-HP: methodology, data curation, reviewing and editing; AW, JS, CRh: conceptualisation, methodology, reviewing and editing; CC, KWi: project administration, reviewing and editing; KG: methodology, reviewing and editing; CM, CRe: conceptualisation, methodology, visualisation, project administration, reviewing and editing. KWa acted as guarantor.

Funding This trial is funded by the NIH National Institute on Aging (R33 AG068945).

Competing interests None declared.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Prevention CfDCa. Flu vaccination coverage, united states, 2022-2023 influenza season. Available: https://www.cdc.gov/flu/fluvaxview/coverage-2223estimates.htm [Accessed 30 Jan 2024 ].

² Patel MS, Volpp KG, Asch DA. Nudge Units to Improve the Delivery of Health Care. N Engl J Med 2018; 378: 214–6. doi:10.1056/NEJMp1712984

³ Hsiao C-J, Hing E, Ashman J. Trends in Electronic Health Record System Use Among Office-based Physicians, United States. Centers for Disease Control and 2007.

⁴ Bhounsule P, Peterson AM. Characteristics of Hospitals Associated with Complete and Partial Implementation of Electronic Health Records. Perspect Health Inf Manag 2016; 13: 1c.

⁵ Wijesundara JG, Ito Fukunaga M, Ogarek J, et al. Electronic Health Record Portal Messages and Interactive Voice Response Calls to Improve Rates of Early Season Influenza Vaccination: Randomized Controlled Trial. J Med Internet Res 2020; 22: e16373. doi:10.2196/16373

⁶ Szilagyi PG, Casillas A, Duru OK, et al. Evaluation of behavioral economic strategies to raise influenza vaccination rates across a health system: Results from a randomized clinical trial. Prev Med 2023; 170: 107474. doi:10.1016/j.ypmed.2023.107474

⁷ Cutrona SL, Golden JG, Goff SL, et al. Improving Rates of Outpatient Influenza Vaccination Through EHR Portal Messages and Interactive Automated Calls: A Randomized Controlled Trial. J Gen Intern Med 2018; 33: 659–67. doi:10.1007/s11606-017-4266-9

⁸ Moniz MH, Hasley S, Meyn LA, et al. Improving influenza vaccination rates in pregnancy through text messaging: a randomized controlled trial. Obstet Gynecol 2013; 121: 734–40. doi:10.1097/AOG.0b013e31828642b1

⁹ López Campos M, Reinao Cegoñino H, Lorente Riverola J, et al. New tools to improve flu vaccination coverage in patients with risk factors. Vacunas (Eng Ed) 2022; 23: 106–12. doi:10.1016/j.vacune.2022.07.009

¹⁰ Johansen ND, Vaduganathan M, Bhatt AS, et al. Nationwide Utilization of Danish Government Electronic letter system for increasing inFLUenza vaccine uptake (NUDGE-FLU): Study protocol for a nationwide randomized implementation trial. Am Heart J 2023; 260: 58–71. doi:10.1016/j.ahj.2023.02.009

¹¹ Baskin E. Increasing influenza vaccination rates via low cost messaging interventions. PLoS One 2018; 13: e0192594. doi:10.1371/journal.pone.0192594

¹² Marshall NJ, Lee JL, Schroeder J, et al. Influence of Digital Intervention Messaging on Influenza Vaccination Rates Among Adults With Cardiovascular Disease in the United States: Decentralized Randomized Controlled Trial. J Med Internet Res 2022; 24: e38710. doi:10.2196/38710

¹³ Szilagyi PG, Duru OK, Casillas A, et al. Text vs Patient Portal Messaging to Improve Influenza Vaccination Coverage: A Health System-Wide Randomized Clinical Trial. JAMA Intern Med 2024; 184: 519–27. doi:10.1001/jamainternmed.2024.0001

¹⁴ Halpern SD, Ubel PA, Asch DA. Harnessing the power of default options to improve health care. N Engl J Med 2007; 357: 1340–4. doi:10.1056/NEJMsb071595

¹⁵ Kahneman D, Knetsch JL, Thaler RH. Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias. J Econ Perspect 1991; 5: 193–206. doi:10.1257/jep.5.1.193

¹⁶ Levin IP, Schneider SL, Gaeth GJ. All Frames Are Not Created Equal: A Typology and Critical Analysis of Framing Effects. Organ Behav Hum Decis Process 1998; 76: 149–88. doi:10.1006/obhd.1998.2804

¹⁷ Loewenstein G, Asch DA, Volpp KG. Behavioral economics holds potential to deliver better results for patients, insurers, and employers. Health Aff (Millwood) 2013; 32: 1244–50. doi:10.1377/hlthaff.2012.1163

¹⁸ Mehta SJ, Day SC, Norris AH, et al. Behavioral interventions to improve population health outreach for hepatitis C screening: randomized clinical trial. BMJ 2021; 373: 1022: n1022. doi:10.1136/bmj.n1022

¹⁹ Mehta SJ, Khan T, Guerra C, et al. A Randomized Controlled Trial of Opt-in Versus Opt-Out Colorectal Cancer Screening Outreach. Am J Gastroenterol 2018; 113: 1848–54. doi:10.1038/s41395-018-0151-3

²⁰ Mehta SJ, McDonald C, Reitz C, et al. A randomized trial of mailed outreach with behavioral economic interventions to improve liver cancer surveillance. Hepatol Commun 2024; 8: e0349. doi:10.1097/HC9.0000000000000349

²¹ Patel MS, Milkman KL, Gandhi L, et al. A Randomized Trial of Behavioral Nudges Delivered Through Text Messages to Increase Influenza Vaccination Among Patients With an Upcoming Primary Care Visit. Am J Health Promot 2023; 37: 324–32. doi:10.1177/08901171221131021

²² Milkman KL, Patel MS, Gandhi L, et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc Natl Acad Sci U S A 2021; 118: e2101165118. doi:10.1073/pnas.2101165118

²³ Mehta SJ, Mallozzi C, Shaw PA, et al. Effect of Text Messaging and Behavioral Interventions on COVID-19 Vaccination Uptake: A Randomized Clinical Trial. JAMA Netw Open 2022; 5: e2216649. doi:10.1001/jamanetworkopen.2022.16649

²⁴ Dai H, Saccardo S, Han MA, et al. Behavioural nudges increase COVID-19 vaccinations. Nature New Biol 2021; 597: 404–9. doi:10.1038/s41586-021-03843-2

²⁵ Asch DA, Volpp KG. On the Way to Health. LDI Issue Brief 2012; 17: 1–4.

²⁶ Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clin Trials 2004; 1: 297–305. doi:10.1191/1740774504cn024oa

²⁷ R: a language and environment for statistical computing. R Foundation for Statistical Computing; 2019.

²⁸ Yu H, Li F, Gallis J, et al. cvcrand: A Package for Covariate-constrained Randomization and the Clustered Permutation Test for Cluster Randomized Trials. R J 2019; 11: 191. doi:10.32614/RJ-2019-027

²⁹ Schultz PW, Nolan JM, Cialdini RB, et al. The constructive, destructive, and reconstructive power of social norms. Psychol Sci 2007; 18: 429–34. doi:10.1111/j.1467-9280.2007.01917.x

³⁰ Liao JM, Yan XS, et al. The Effect Of Clinician Feedback Interventions On Opioid Prescribing: Study examines the effect of clinician feedback interventions on opioid prescribing. Health Aff 2022; 41: 424–33. doi:10.1377/hlthaff.2021.01407

³¹ Patel MS, Kurtzman GW, Kannan S, et al. Effect of an Automated Patient Dashboard Using Active Choice and Peer Comparison Performance Feedback to Physicians on Statin Prescribing: The PRESCRIBE Cluster Randomized Clinical Trial. JAMA Netw Open 2018; 1: e180818. doi:10.1001/jamanetworkopen.2018.0818

³² Lu D, Qiao Y, Brown NE, et al. Racial and Ethnic Disparities in Influenza Vaccination among Adults with Chronic Medical Conditions Vary by Age in the United States. PLoS One 2017; 12: e0169679. doi:10.1371/journal.pone.0169679

³³ Lu P, O’Halloran A, Williams WW, et al. Racial and ethnic disparities in vaccination coverage among adult populations in the U.S. Vaccine (Auckl) 2015; 33: D83–91. doi:10.1016/j.vaccine.2015.09.031

³⁴ Jain A, van Hoek AJ, Boccia D, et al. Lower vaccine uptake amongst older individuals living alone: A systematic review and meta-analysis of social determinants of vaccine uptake. Vaccine (Auckl) 2017; 35: 2315–28. doi:10.1016/j.vaccine.2017.03.013

³⁵ Kohlhammer Y, Schnoor M, Schwartz M, et al. Determinants of influenza and pneumococcal vaccination in elderly people: a systematic review. Public Health 2007; 121: 742–51. doi:10.1016/j.puhe.2007.02.011

³⁶ Bruno J, Johnson S, Hesley J. Robotic disruption and the new revenue cycle: robotic process automation represents an immediate new opportunity for healthcare organizations to perform repetitive, ongoing revenue cycle processes more efficiently and accurately. Healthcare Fin Manag 2017 2017; 54.

³⁷ Patel MS, Volpp KG, Small DS, et al. Using Active Choice Within the Electronic Health Record to Increase Influenza Vaccination Rates. J Gen Intern Med 2017; 32: 790–5. doi:10.1007/s11606-017-4046-6