Applying improvement science to establish a resident sustained quality improvement (QI) educational model

Abstract

Background

Prior to 2017, internal medicine (IM) residents at the University of Alberta did not have a standardised quality improvement (QI) educational curriculum. Our goal was to use QI principles to develop a resident sustained curriculum using the Evidence-based Practice for Improving Quality (EPIQ) training course.

Methods

Three one-year Plan–Do–Study–Act (PDSA) cycles were conducted. The EPIQ course was delivered to postgraduate year (PGY) 1–3 residents (n=110, PDSA 1) in 2017, PGY-1 residents (n=27, PDSA 2) in 2018 and PGY-1 residents (n=28, PDSA 3) in 2019. Trained residents were recruited as facilitators for PDSA 2 and 3. Residents worked through potential QI projects that were later presented for evaluation. Precourse and postcourse surveys and tests were conducted to assess knowledge acquisition and curriculum satisfaction. Process, outcome and balancing measures were also evaluated.

Results

In PDSA 1, 98% felt they had acquired understanding of QI principles (56% increase), 94% of PGY-2 and PGY-3 residents preferred this QI curriculum compared with previous training, and 65% of residents expressed interest in pursuing a QI project (15% increase). In PDSA 2, tests scores of QI principles improved from 77.6% to 80%, and 40% of residents expressed interest in becoming a course facilitator. In PDSA 3, self-rated confidence with QI methodology improved from 53% to 75%. A total of 165 residents completed EPIQ training and 11 residents became course facilitators.

Conclusions

Having a structured QI curriculum and working through practical QI projects provided valuable QI training for residents. Feedback was positive, and with each PDSA cycle there was increased resident interest in QI. Developing this curriculum using validated QI tools highlighted areas of change opportunity thereby enhancing acceptance. As more cycles of EPIQ are delivered and more residents become facilitators, it is our aim to have this curriculum sustained by future residents.

Correspondence to Dr Caitlyn Collins; [email protected]

Problem

As our healthcare system evolves, it has become increasingly important for residents to participate in quality improvement (QI).^1–3 Residency training focuses primarily on acquiring medical knowledge and clinical application; however, minimal instruction is on QI. Participation in QI is now an accreditation standard and part of the Canadian Royal College of Physicians and Surgeons Certification in Internal Medicine (IM).⁴ There are several well-designed approaches to establish a QI curriculum described in literature.⁵ However, we are unaware of a postgraduate medical education QI model that develops a curriculum sustained by residents.

Prior to 2017, IM residents at the University of Alberta did not have a standardised QI curriculum. Previous QI training consisted of residents completing Institute for Healthcare Improvement online modules and then presenting individual QI ideas to a panel of faculty physicians with limited engagement.⁶ The aim of this study was to apply improvement science to develop and implement a standardised resident sustained QI educational curriculum for the IM program.

Approach

Our study team was comprised of the following members: a postgraduate year (PGY) 1 and PGY-3 resident, two faculty physicians and a QI consultant. We used the Model of Improvement supported by iterative Plan–Do–Study–Act (PDSA) cycles to guide knowledge translation along with practical experience to create continuous improvement.⁷ The Donabedian evaluation model was used, which provided a set of measures to determine improvement.⁸ The participants in our study were PGY-1 to PGY-3 IM residents.

A literature review, along with QI tools such as process mapping and an Ishikawa diagram (figure 1), were used to identify current curriculum barriers and areas for improvement. The main gaps identified were the lack of hands-on application of QI tools, limited faculty physicians to support QI teaching, limited connection to organisational QI projects, independent time needed for QI learning and no QI consultant resource to support physicians/residents with real-time QI coaching.

Figure 1. Ishikawa diagram of root cause analysis that highlights gaps in the current QI educational approach. The process maps outline the current steps involved and the proposed new process steps to close the gaps identified. AHD, academic half day; EPIQ, Evidence-based Practice for Improving Quality; IHI, Institute for Healthcare Improvement; PDSA, Plan-Do-Study-Act; PGME, postgraduate medical education; QI, quality improvement.

To close the gaps identified, we chose the 4-hour Evidence-based Practice for Improving Quality (EPIQ) workshop as the QI learning platform. This workshop facilitates team-based learning by having small groups work through 10 sequential steps of project development, combining both didactic and hands-on learning techniques to develop PDSA cycles of rapid change.^{9 10}

The objectives were to train IM residents through the EPIQ workshop leading to completion of Aim and Change forms, identify residents to facilitate future EPIQ workshops and align interested residents to active clinical QI projects. The QI consultant was the lead instructor for the workshop and coached participants on QI projects. Faculty physicians were simultaneously invited to train in the EPIQ workshop. The study measures used to determine if there were improvements were the process measures (what we are going to do), the outcome measures (what we hope to achieve) and the balancing measures (what we do not want to negatively impact) as outlined in table 1.

Table 1

QI study measures

Process	Outcome	Balancing
# of completed Aim and Change forms at the end of the EPIQ workshop. # of residents that presented their QI project ideas.	# of residents that completed the EPIQ workshop. # of residents that participated in active QI projects with faculty physicians. # of residents that became EPIQ facilitators. # of residents and faculty that became lead EPIQ instructors.	# of hours spent on QI training.

EPIQ, Evidence-based Practice for Improving Quality; QI, quality improvement.

QI educational model implementation

We completed three PDSA cycles over three years to implement our proposed QI educational curriculum. The approach taken for each PDSA cycle is outlined below (figure 2).

Figure 2. Establishment of a sustained resident and physician QI educational pipeline provided the framework for iterative cycles of training, recruitment and resident/physician opportunity to participate and lead organisational clinical QI projects. PGY, postgraduate year; PDSA, Plan-Do-Study-Act; QI, quality improvement.

Methods

PDSA 1

PDSA 1 consisted of delivering EPIQ workshops to PGY-1 to PGY-3 residents (n=110) in November 2017. During designated academic time, residents were assigned to groups of six and instructed by the QI consultant. Each resident group had a selected facilitator who were members of the study team. The residents brainstormed potential QI ideas and applied the 10 steps of the EPIQ platform to develop hypothetical projects. This process included problem identification, root cause analysis, intervention determination, process mapping, identifying measures and completing an Aim and Change form. Examples of QI project topics are listed in box 1. Following completion of the course, resident groups delivered formal presentations of their QI projects in April 2018, which were evaluated by the study team. Postcourse surveys were sent to residents via email to evaluate knowledge acquisition of QI principles, likelihood to take part in future QI projects, interest in learning more about QI and interest in becoming an EPIQ facilitator.

Box 1

Resident group quality improvement (QI) ideas per Plan–Do–Study–Act (PDSA) cycle

QI project topics used during Evidence-based Practice for Improving Quality workshop

PDSA 1:

• Improving daily weights measurement.

• Early transition from intravenous to oral antibiotics.

• Improving goals of care (GOC) documentation.

• Finetuning handover processes.

• Earlier discontinuation of foley catheters.

• Decreasing use of unnecessary intravenous maintenance fluids.

• Decreasing routine ordering of creatinine for patients with end-stage renal disease on dialysis.

PDSA 2:

• Chronic steroid prophylaxis – overlooked by clinicians?

• Handover: improving a hectic and stressful process.

• Improving the recording of daily weights for heart failure patients.

• Targeting incomplete bowel preparations for inpatient colonoscopies.

• Universal application of CAM (confusion assessment method) tool to screen for delirium among hospitalised patients.

• Improving GOC documentation.

PDSA 3:

• Improving GOC documentation in oncology patients.

• Streamlining resident handover processes.

• Early discontinuation of foley catheters.

• Improving stat blood work ordering in new electronic medical record system.

• Decreasing overuse intravenous maintenance fluids on the wards.

• Improving communication between hospital physician and primary care physician at time of hospital discharge.

PDSA 2

In PDSA 2 study team members delivered the EPIQ workshop to PGY-1 residents (n=27) in December 2018. In this second iteration, residents completed both precourse tests and surveys at the start of the workshop and postcourse tests and surveys immediately concluding the workshop. All surveys and tests were anonymised. Resident facilitators recruited from PDSA 1 assisted during the workshop to support resident group exercises. Residents presented their QI projects in May 2019.

PDSA 3

This PDSA cycle was initiated in November 2019, and the workshop was held for the PGY-1 residents (n=28). The same strategy was undertaken as in PDSA 2 except that the timing between the EPIQ workshop and resident presentations decreased based on feedback from PDSA 2. Resident presentations were in January 2020.

Results

PDSA 1

Forty-seven per cent (52/110) of residents completed the postcourse survey and the responses are compiled in table 2. Additionally, the PGY-2 and PGY-3 residents were also asked whether they preferred the new EPIQ workshop and presentations compared with previous QI training, and there was a section for comments and feedback.

Table 2

Results by PDSA cycle

Pre-EPIQ	PDSA 1	PDSA2	PDSA3
	Total yes responses	Total yes responses	Total yes responses
Do you feel you have a clear understanding of what quality improvement (QI) is?	22/52 (42%)	6/25 (24%)	12/28 (43%)
Do you feel comfortable taking on a QI project?	13/52 (25%)	6/25 (24%)	9/28 (31%)
Are you interested in QI?	26/52 (50%)	22/25 (88%)	20/28 (71%)
Are you currently involved in QI or a QI project?	15/52 (29%)	1/25 (4%)	2/28 (7.1%)
	PDSA 1	PDSA2	PDSA3
Post-EPIQ	Total yes responses	Total yes responses	Total yes responses
Do you have basic understanding of QI principles?	51/52 (98%)	25/25 (100%)	24/24 (100%)
Do you feel comfortable working through the EPIQ 10 steps?	46/52 (88.4%)	20/25 (80%)	22/24 (92%)
Are in interested in pursuing a QI project?	34/52 (65.3%)	21/25 (84%)	16/24 (67%)
Are you interested in becoming a QI EPIQ facilitator?	9/52 (17.3%)	10/25 (40%)	5/24 (21%)
For PGY-2 and PGY-3: do you feel the EPIQ workshop and presentation is preferable to the previous QI curriculum for the purposes of resident level QI education? (Reminder: previous curriculum was to create and present a QI project on your own)	33/35 (94.2%)	Not applicable	Not applicable
Pre-EPIQ average test scores	Not completed	7.76/10	7.43/10
Post-EPIQ average test scores	Not completed	8/10	7.92/10
Pre-EPIQ self-rated confidence matrix with QI principles (score/5)	Not completed	2.73	2.67
Post-EPIQ self-rated confidence matrix with QI principles (score /5)	Not completed	3.78	3.79

EPIQ, Evidence-based Practice for Improving Quality; PGY, postgraduate year; QI, quality improvement.

Based on participant responses, residents felt that they had an increase in basic QI knowledge (56%), interest in pursuing a QI project (15%) and interest in facilitating EPIQ workshops (17%). Ninety-four per cent (33/35) of PGY-2 and PGY-3 residents preferred this new QI curriculum to previous QI training. Sixty-three per cent of residents enjoyed the stepwise approach to learning QI, and fifty per cent of residents indicated the QI project presentations were too lengthy.

PDSA 2 and 3

Results from the postcourse survey showed that all residents (100%) indicated they had a basic understanding of QI principles after the EPIQ workshop compared with the precourse survey, 76% increase in PDSA 2 and 57.1% in PDSA 3 (table 2). Scores compared from the pretest and post-test showed QI knowledge improved (table 2). Furthermore, self-rated confidence with QI principles improved from 54.6% to 75.6% in PDSA 2 and from 53.4% to 75.8% in PDSA 3 (table 2).

Results summary

To determine if curriculum changes resulted in an improvement, 100% of residents completed their Aim and Change forms and 100% of residents completed project presentations (process measures). Over 3 years, a total of 165 IM residents completed the EPIQ workshop, 30 residents have been part of active QI projects and 10 have presented (poster or oral) at national and international conferences. Several residents (11) have become EPIQ workshop facilitators and one resident has become a lead EPIQ instructor.

Discussion

Improvement science is a burgeoning concept that describes how to improve and make changes effectively, systematically examining the methods and factors that facilitate QI.¹¹ The pragmatic application of improvement science supported the review of the QI training curriculum from the perspective and experience of residents. The knowledge gained was used to establish a clear aim, to define measurements aiding in understanding how change occurred and to identify actions that were tested using iterative change cycles.

Each PDSA cycle provided lessons learned that allowed the authors to make subsequent changes to each cycle. In PDSA 1, a postcourse survey was distributed via email with a low response rate of 47% (52/110) despite several reminder emails. Because there was no precourse survey or test, there was no real-time assessment of knowledge. For PDSA 2 and 3, residents completed the precourse and postcourse tests and surveys on the workshop day. The response rates improved to 93% (25/27) in PDSA 2 and 86% (24/28) in PDSA 3. By implementing a test along with the survey, this provided comparative data about knowledge acquisition versus training perception. The test scores improved from the precourse to postcourse test, however not substantially. We hypothesise that the minimal increase was likely secondary to the low number of questions on the test (10 total), which created little spread in the data.

In PDSA 1, all three resident years (PGY-1 to PGY-3) completed the EPIQ course. In PDSA 2 and 3, there was only one resident year (PGY-1) taking the course that allowed for smaller group size. Throughout the three PDSA cycles, feedback on course satisfaction and suggested areas for improvement was collected. After PDSA 1, many residents noted that the presentation day was too long. For PDSA 2 and 3, this issue was addressed as there were fewer residents in the workshop. There was also feedback that the time to brainstorm potential QI initiatives was limited, so for future years we plan to ask residents to start brainstorming ideas the month prior to the workshop. Overall, feedback obtained from residents has been overwhelmingly positive. Based on the data collected, this QI curriculum seems to be efficient with reducing QI training time and yet enabling residents to acquire foundational QI knowledge.

Interested residents that have completed the workshop are being linked with active QI teams, and enough resident facilitators have been identified to continue delivering the EPIQ workshop in future years, which further increases resident learning of QI. The IM program now elects a QI representative (PGY-2 resident) each year to coordinate delivery of the EPIQ workshop and presentation day. As this curriculum continues to be delivered and more residents participate in QI, this will cultivate an atmosphere within our healthcare system to create ongoing positive change.

University of Alberta Internal Medicine residency program.

Footnote

Contributors CC is the corresponding author. CC and PM are credited for conception, design, data collection, data analysis and creation of manuscript. AT oversaw the design of the study. SI, NK and AT critically revised/reviewed the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement All data relevant to the study are included in the article.

References

¹ Ogrinc G, Headrick LA, Morrison LJ, et al. Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med 2004; 19: 496–500. doi:10.1111/j.1525-1497.2004.30102.x http://www.ncbi.nlm.nih.gov/pubmed/15109311

² Tess AV, Yang JJ, Smith CC, et al. Combining clinical microsystems and an experiential quality improvement curriculum to improve residency education in internal medicine. Acad Med 2009; 84: 326–34. doi:10.1097/ACM.0b013e31819731bf http://www.ncbi.nlm.nih.gov/pubmed/19240439

³ Bonnes SL, Ratelle JT, Halvorsen AJ, et al. Flipping the quality improvement classroom in residency education. Acad Med 2017; 92: 101–7. doi:10.1097/ACM.0000000000001412 http://www.ncbi.nlm.nih.gov/pubmed/27680317

⁴ Wong RY. Teaching quality improvement in residency education. The Royal College of Physicians and Surgeons, 2015. Available: http://www.royalcollege.ca/rcsite/documents/cbd/teaching-quality-improvement-e.epub [Accessed Mar 2020 ].

⁵ Wong BM, Goldman J, Goguen JM, et al. Faculty-Resident "Co-learning": A Longitudinal Exploration of an Innovative Model for Faculty Development in Quality Improvement. Acad Med 2017; 92: 1151–9. doi:10.1097/ACM.0000000000001505 http://www.ncbi.nlm.nih.gov/pubmed/28746138

⁶ Institute for Healthcare Improvement. Ready to start improving? Available: http://www.ihi.org/education/IHIOpenSchool/Pages/default.aspx [Accessed Mar 2020 ].

⁷ Berwick DM. A primer on leading the improvement of systems. BMJ 1996; 312: 619–22. doi:10.1136/bmj.312.7031.619 http://www.ncbi.nlm.nih.gov/pubmed/8595340

⁸ Ayanian JZ, Markel H. Donabedian's lasting framework for health care quality. N Engl J Med 2016; 375: 205–7. doi:10.1056/NEJMp1605101 http://www.ncbi.nlm.nih.gov/pubmed/27468057

⁹ EPIQ Workshop Development Committee. Evidence-Based Practice for Improving Quality (EPIQ) workshop. Canadian neonatal network, Toronto, 2018. Available: http://www.epiq.ca [Accessed Mar 2020 ].

¹⁰ Lee SK, Aziz K, Singhal N, et al. The Evidence-based Practice for Improving Quality method has greater impact on improvement of outcomes than dissemination of practice change guidelines and quality improvement training in neonatal intensive care units. Paediatr Child Health 2015; 20: e1–9. http://www.ncbi.nlm.nih.gov/pubmed/25722645

¹¹ Health Foundation. Evidence Scan: Improvement Science, 2011. Available: https://www.health.org.uk/sites/default/files/ImprovementScience.pdf [Accessed Dec 2020 ].