SEIPS 101 and seven simple SEIPS tools

Correspondence to Dr Richard J Holden, Indiana University, Indianapolis, Indiana, USA; [email protected]

In the past 15 years, SEIPS (Systems Engineering Initiative for Patient Safety)^1–3 and related conceptual models^{4 5} were developed to study and improve healthcare. These theoretical models depict how work systems affect health-related outcomes, such as patient safety, and can be used to guide research and improvement efforts. Various versions of the SEIPS model have been used by academics and practitioners, but a recent review argued that broader use and benefits can be achieved through ‘an easy-to-use version of the model and simplified tools for model application’.⁶ This concords with repeated requests we have received for simple, practical tools to apply SEIPS and with the general critique regarding the dearth of easy to use systems engineering tools.⁷

Accordingly, we offer SEIPS 101, a simplified, practice-oriented SEIPS model meant for easy use by practitioners, researchers and others, regardless of prior familiarity with SEIPS. We also for the first time present seven simple SEIPS tools virtually anyone can use off-the-shelf.

The SEIPS model

The SEIPS model is a theoretical model rooted in human-centred systems engineering or ‘human factors/ergonomics’.^{8 9} All versions of the model depict three major components, the work system, processes and outcomes; key characteristics or factors of each; and how the components affect one another. The SEIPS model has been used to understand or design sociotechnical systems and has supported evaluation, planning and research activities. The first version of the SEIPS model was published in 2006 in this journal by Carayon and colleagues,¹ based on work dating to the 1980s.^10–12 The next addition to the SEIPS family was SEIPS 2.0, proposed by Holden et al ³ primarily to address the work done by patients, families and other non-professionals. SEIPS 2.0 made theoretical expansions to the work system, processes and outcomes components and introduced the configural diagram tool. Carayon et al’s² SEIPS 3.0 was subsequently published to further elaborate the processes component and promote attention to the patient journey as it unfolds over time and space.

SEIPS 101

Unlike its predecessors, the SEIPS 101 model (figure 1) does not expand the theory of sociotechnical systems in healthcare. Instead, it is a simplified, practically minded sketch of the most essential SEIPS components. SEIPS 101 is designed to be streamlined, memorable and thus easier to understand, replicate and use. The SEIPS 101 model retains the three major SEIPS components, represented by unique shapes in the figure: work systems (square); work processes (triangle) and work outcomes (circle). Work systems are comprised of interacting structural elements that together produce performance. Every work system minimally has the components people, environments, tools and tasks, whose first letters spell ‘PETT’. The environments, fully described in other SEIPS models, are physical, socio-organisational and external. The physical environment refers to physical layout, location and factors such as lighting, noise and temperature. The socio-organisational environment describes the attributes of an organisational unit (eg, a hospital, department, clinic, home or programme) such as structure, procedures, roles and responsibilities, relationships and organisational culture. The external environment is that which affects the unit of interest from outside, for example, the regulatory, legal, economic, political, cultural or societal contexts.

Figure 1. SEIPS 101 simplified model. Work systems are depicted by a square, evoking the iconic ‘connected boxes’ visualisation of person-centred systems: an interconnected set of factors with people in the centre. Work systems shape and produce work processes, depicted as a triangle to convey flow from left-to-right. The end results are work outcomes, drawn as a circle for the ‘O’ in outcome. Arrows represent causal feedback loops. Several details are omitted for simplicity. SEIPS, Systems Engineering Initiative for Patient Safety.

Work processes are how the work is done and how it flows. Work processes are physical, cognitive, social-behavioural or a combination.⁴ They can be performed by healthcare professionals, patients and families or collaboratively between professionals and nonprofessionals. Work outcomes result from work systems and work processes. These are desirable or undesirable, distal or proximal. They affect professionals, patients/families or the organisation. Arrows between systems, processes and outcomes represent causal feedback loops.

Using SEIPS 101

SEIPS 101 can be used in projects as a theoretical or practical framework to guide activity. All parts of SEIPS 101 (people, environments, tools, tasks, processes and outcomes) can be observed, measured, analysed and manipulated, though how this is done will vary case-by-case.¹³ A project or analysis should address each part but not necessarily every variation or combination. For example, a hospital service may be characterised by hundreds if not thousands of people, environments, tools, tasks, processes and outcomes. The millions of combinations of these components are unrealistic to depict, analyse or design, so the goal should be to attend as much as possible to the relevant people, environments, tools, tasks, processes and outcomes. A deceptively simple use of SEIPS 101 is drawing it (see online supplemental appendix A), because visual representations are powerful for conveying meaning, convincing others, memorising and achieving common understanding.^{14 15} One might draw the SEIPS 101 model to orient their team to the essential SEIPS components that will be considered in a project or to explain how changing downstream outcomes requires the redesign of upstream work systems and processes.

Seven simple SEIPS tools

To aid further practical applications of SEIPS, we offer and illustrate seven simple SEIPS tools. Based on our and others’ work with SEIPS, we selected tools we could easily describe and illustrate. We also selected tools both novices and experts in systems engineering, practitioners and researchers, could use ‘off the shelf’. As with any tool, collaboration or consultation with systems engineering experts and experience and skills gained using these tools in practice, can enhance efficiency and effectiveness. The tools can be used alone or in any combination. For each tool, we state the SEIPS model components to which it relates and its primary uses. It often helps to adapt these tools to specific projects—another time when tool users may benefit from expert involvement—as seen in illustrative cases below and other published examples.^{16 17} For instance, Zisberg et al implemented an in-hospital mobility intervention by adapting SEIPS 2.0 to: define the primary outcome of interest; study people (eg, attitude) and whole-system factors associated with in-hospital mobility; develop and adapt an intervention strategy; and implement the intervention.¹⁷

Tool #1: PETT scan

The PETT scan (box 1) is a checklist and documentation tool to ensure one considers the full breadth of the work system, namely its people, environments, tools and tasks. Moreover, PETT scans are commonly used to indicate the presence of barriers or facilitators for each of the PETT components or for component-component interactions. Barriers and facilitators are aspects of the components or their interactions that either hinder or support the people in the work system, for example, their ability to do their ‘job’ or interact with other care team members or their experience (eg, satisfaction, frustration) with the work.^18–20 One can quantify barriers and/or facilitators for a given case by assigning each observed barrier/facilitator to one or more PETT categories. These can be visualised as counts or proportions (eg, as a radar chart depicting one’s ‘barrier profile’²¹). Table 1 illustrates how the PETT scan was used in prior studies to analyse the systems barriers and facilitators shaping patient work (abdominal surgical patients’ transition-related work), clinician work (nurses’ performance of their jobs) and collaborative work (family engagement in paediatric hospital rounds).^22–24 A template for the PETT scan is provided in the online supplemental file.

Box 1

PETT scan

• SEIPS components addressed: the work system component, that is, the people, environments, tools, tasks and the interactions between them.

• Primary uses: to describe or design the interacting parts of the entire sociotechnical system, especially when attempting to simply depict a complex system.

PETT, people, environments, tools and tasks; SEIPS, Systems Engineering Initiative for Patient Safety.

Table 1

Examples of PETT scan

Work system factors	Example: patient work System factors associated with transitions and rehospitalisations among patients discharged following abdominal surgery*		Example: clinician work System factors associated with tele-ICU nurses’ job performance†		Example: collaborative work System factors associated with family engagement in the paediatric hospital bedside rounding process‡
	Barriers	Facilitators	Barriers	Facilitators	Barriers	Facilitators
People Patients Healthcare professionals Others	Poor understanding of what would be needed once back at home			Sharing nursing knowledge with bedside ICU nurses	Lack of communication skills of clinicians Parent fatigue	Parent knowledge of their child’s condition
Environments Physical Socio-organisational External		Collaboration from clinician: follow-up call after discharge to help patient with recovery at home	Lack of acceptance of tele-ICU by ICU staff	Positive teamwork and collaboration between tele-ICU and ICU Quiet work environment	Interruptions and noise affecting team communication
Tools	Too many educational materials		Too many logins in multiple health information systems	Access to comprehensive information on patient	Computer as a physical barrier to communication	Use of computer to present and share visual information such as X-ray
Tasks	Receiving inadequate or incomplete instructions about patient care at home		Missing direct patient care in the ICU	Challenging and interesting job content because dealing with various ICU patient problems		Introduction of all team members and their roles
Interactions between people, environments, tools and tasks	Negative interaction in the discharge process: patients receiving insufficient instructions (tasks) in a hurried manner (environments), therefore not understanding what will be needed for home recovery (people)			Positive interaction between tele-ICU and ICU (organisational environment) facilitates communication and sharing of information (tasks)	High clinician workload (environments) may limit their availability and participation in bedside rounding, therefore affecting information exchange and communication (tasks)

*Adapted from Acher et al’s study of system factors contributing to readmissions of surgical patients.²²

†Adapted from Hoonakker et al’s study of tele-ICU nurses.²⁴

‡Adapted from Carayon et al’s study of family engagement in bedside rounds in a paediatric hospital.²³

ICU, intensive care unit; ;PETT, people, environments, tools and technologies.

Overall, the PETT scan is a flexible tool and can be used for at least the following:

• Project planning: to know which factors to consider.

• Intervention design: to know which factors to address.

• Intervention implementation: to anticipate how changes affect all work system factors.

• Intervention evaluation: to assess effects on an array of factors.

• Data collection: to specify methods for capturing each factor.

• Analysis: to classify data into categories of factors.

• Reporting: to present results according to the factors.

• Prioritising: to select the priority order among the various factors.

Tool #2: people map

The people map (box 2) represents the various people involved in a work system and how they relate or interact in practice, not as idealised on an organisational chart. Because people perform tasks and roles, a people map can also show which people or groups perform which activities and how they interact with each other in a role network.^{25 26} People maps may use:

• Shapes to represent roles (eg, patient, pharmacist), individuals (eg, Patient 1, Patient 2) or types (eg, spouse, child).

• Shape size to represent relative frequency or importance.

• Proximity to represent geographic/social distance or groupings.

• Lines or arrows to represent relationships and interactions between people.

Box 2

People map

• SEIPS components addressed: the people component of the work system and to a lesser extent the tasks those people perform.

• Primary uses: to describe the various people involved and how they interact or relate to one another, especially to reveal insights about the properties of social networks (eg, the centrality of a given role) or to contrast distinct ‘types’ of person or groups of people (ie, personas).

SEIPS, Systems Engineering Initiative for Patient Safety.

Figure 2 offers example people maps based on heart failure care research.^{21 27–35} Figure 2A simply depicts people on a clinical care team, whereas figure 2B shows how those individuals interact, revealing key insights such as the care coordinator’s central role as an information hub. Figure 2C shows how distinct types of people or systems—called ‘personas’^{27 28}—can be shown on a people map to highlight key differences, for example, between patients with a full household but few outside friends versus patients living alone but surrounded by many distant friends and family. Despite the simplicity of this example, typical evidence-based personas can depict far more nuanced differences. Such personas become tools for designers to both understand the range of stakeholders or situations to be accommodated and accommodate each.³⁶ For example, designers creating a technology or instructions for posthospitalisation self-care, may be influenced by the personas in figure 2C to design features for patients in both dense households and living alone.

Figure 2. Example people maps. CC, care coordinator; Cg, caregiver; NP, nurse practitioner; P, patient; PCP, primary care physician; Ph, community pharmacist; RN, registered nurse; S, specialty care physician.

Tool #3: tasks and tools matrices

The tasks and tools matrices (box 3) describe the work system’s tasks, tools and task-tool interactions, illustrated in table 2 with results from an analysis of a care coordination programme.³⁷ The tasks matrix lists and describes key tasks along the dimensions of who, why, how often, how and when each task is performed. Other dimensions (eg, the task’s criticality) can be added. The tools matrix documents key artefacts, instruments or technologies in the system and their users, purpose, use frequency, accessibility and estimated or measured³⁸ usability. Tasks and tools can be ordered or organised, for example, by frequency, criticality or timeline (eg, before, during, after a home visit).³⁷ The tasks X tools matrix depicts which tools are used for which tasks. Completing these matrices informs changes to tasks (eg, reassigning scheduling or having caregivers self-administer assessments) and tools (eg, replace handheld scanner with smartphone or tablet). Templates of these matrices are available in the online supplemental file.

Table 2

Examples of tasks, tools and tasks X tools matrices based on a prior work system analysis of a community-based brain health care coordination programme³⁷

(A) Tasks matrix
	Who performs	Goal(s) of task	Frequency	How performed	When performed	Notes
Appointment scheduling	CCA, SW	Arrange home visits	Daily	Staff calls, records in calendar	Before home visit	Assign to dedicated scheduler?
Health assessment	CCA, RN	Deliver right care	Every 2 weeks	CCA administers instruments	At home visit	Can be done by CG?
Relationship building	CCA, SW, RN, CG	Establish trust	As needed	Listening, humour, show interest, offer help	Before, at or after home visit	Takes most time, critical to success

(B) Tools matrix
	Users	Purpose of use	Frequency of use	Ease of access	Usability	Notes
Smartphone	CCA	Communication, navigation	High	High	High	Staff like to use
Paper assessments	CCA, RN	Collect and monitor health	High	Low	High	Get lost, damaged
Handheld scanner	CCA, RN, SW	Digitise paper assessments	Low	Medium	Low	Staff hate to use

(C) Tasks X tools matrix
	Scheduling		Health assessment		Relationship building
Smartphone	–		–		Yes
Paper assessments	–		Yes		–
Handheld scanner	–		Yes		–

CCA, care coordination assistant; CG, informal caregiver; RN, registered nurse; SW, social worker.

Box 3

Tasks and tools matrices

• SEIPS components addressed: The tasks and tools components of the work system.

• Primary uses: to enumerate, describe and evaluate tasks and tools, often to design or redesign them or to identify gaps between the tasks performed and the tools available.

SEIPS, Systems Engineering Initiative for Patient Safety.

Tool #4: outcomes matrix

An outcomes matrix (box 4) identifies and organises the various outcomes of interest, whether they represent project goals, measures to be collected or evaluation criteria. Not every outcome needs to be depicted or measured but a thorough analysis of outcomes for various stakeholders, including attention to proximal and distal as well as desirable and undesirable outcomes,³ can be useful for planning or evaluation. In light of our human-centred systems approach based on human factors and ergonomics,³⁹ it is important to consider outcomes for various stakeholders and to achieve an optimal balance of outcomes across stakeholders, such as patients, informal caregivers, clinicians and healthcare organisations. For instance, a project aimed at improving an infection prevention practice should achieve improvement in patient safety (eg, reduction of hospital-acquired infections) while avoiding additional stress and negative outcomes for physicians and nurses (eg, due to additional steps or tasks and increasing burden). Therefore, we recommend that outcomes be balanced across all key stakeholders relevant for the particular project or study. Outcomes can also be ranked or rated on priority, likelihood or other dimensions and their measures specified.¹⁶ A template outcomes matrix is provided in the online supplemental file.

Box 4

Outcomes matrix

• SEIPS components addressed: the outcomes component.

• Primary uses: to proactively or reactively identify the relevant outcomes to consider, especially when needing to comprehensively document multiple types of outcomes and their measures in an organised manner.

SEIPS, Systems Engineering Initiative for Patient Safety.

Tool #5: journey map

The journey map (box 5) or process map is a tool to explain one or more work processes while simultaneously depicting other relevant factors or conditions over time. Figure 3 has examples of simplified journey maps: one for a patient experiencing a change in their medications⁴⁰ (figure 3A) and one for bedside rounding in a paediatric hospital⁴¹ (figure 3B). There are many tools for depicting process or workflow, with great variation in what is presented and how this is done.^42–44 For example, some process maps contain merely the steps in a process or a list of activities organised by their goals and subgoals. A journey map is a popular variety of process map very well aligned with SEIPS concepts.² The journey map can be used as a tool to represent how people interact dynamically with other people, tasks, tools and environments over time. Journey maps can depict touchpoints in the process where interactions occur, feelings elicited by the interactions, barriers or ‘pain points’ that affect the journey, alternative paths or variations and changing conditions or relationships (eg, demands vs capacity). By leveraging colour, imagery, spatial relationships and other visualisation techniques, journey maps often convey multidimensional information in a salient, usable and memorable way.⁴⁵

Box 5

Journey map

• SEIPS components addressed: the process component, often simultaneously with work system and outcomes components.

• Primary uses: to depict a process over time and how work system conditions and outcomes change during the course of the process, often used to identify problems or patterns that will need to be addressed when modifying or designing a process or system.

SEIPS, Systems Engineering Initiative for Patient Safety.

Figure 3. Example journey maps.

Tool #6: interactions diagram

This tool (box 6)—also called the configural diagram^{3 13 29 46}—depicts how work system factors interact. Showing these interactions helps explain ‘why’ something happened or is happening, from a whole-systems perspective. Interactions diagrams are not meant to be fully inclusive; they are better suited to show only the most relevant or consequential factors and interactions. Figure 4 applies the interactions diagram to show how multiple factors combine and affect or mitigate each other to shape nutrition for hypertensive patients living in food deserts. Even in the face of seemingly uncontrollable factors (eg, where patients dwell), the system can result in good performance, depending on the interactions of all relevant system factors (eg, programmes to screen for and address food insecurity). Similar diagrams could be drawn to describe whole-system interactions associated with episodic or long-term outcomes at different levels of analysis, for example, a single patient’s exercise behaviour, staff burnout in a hospital unit or quality outcomes across the state’s primary care clinics.

Box 6

Interactions diagram

• SEIPS components addressed: the interactions in the work system component.

• Primary uses: to depict the relevant subset of work system factors whose interactions are meaningful, often with the intent to select the aspects of a system to address during design. A second use is to draw comparisons of two or more sets of system interactions.

SEIPS, Systems Engineering Initiative for Patient Safety.

Figure 4. Example interactions diagram.

Another use of interactions diagrams is comparing the contributing system factors between:

• Settings: for example, hospitals, units, regions, clinics differing in size.

• Populations: for example, by age, condition, race/ethnicity.

• Teams: for example, interprofessional versus not, Red Team versus Blue Team.

• Processes: for example, medication ordering versus dispensing versus administration.

• Outcomes: for example, high versus low performance, harm versus no harm.

• Time: for example, years, seasons, pre-post an intervention.

• Systems or approaches: for example, Design 1 versus Design 2, Approach A versus B.

Tool #7: systems story

Storytelling is a tool pervading time and culture. Stories are compelling and easy to understand, remember, reshare and repurpose, yet convey much information and complexity.⁴⁷ The systems story tool (box 7) is a story frame (or logic model) about how things happen in systems according to SEIPS: how the system’s design produces changes in processes, thus resulting in different outcomes. This tool is often used to frame a persuasive or memorable argument about how work systems, processes and outcomes are related. An example systems story contrasts hospital systems’ responses to the COVID-19 crisis:^{48 49}

• Militaristic mindset: the system’s blame culture and mistrust of frontline clinicians produced top-down, centralised decision-making processes, causing outcomes of stress and less creative solutions.

• Agile mindset: the system’s agile culture and trust in frontline clinicians empowered local teams to employ innovative, experiment-based processes, producing outcomes of time savings and innovative solutions.

Box 7

Systems story

• SEIPS components addressed: the combination of work system, processes and outcomes.

• Primary uses: to frame a story about how work systems, processes and outcomes are related, often to make a persuasive or memorable argument to one’s audience.

SEIPS, Systems Engineering Initiative for Patient Safety.

Once the systems story is framed, telling the story is also important; one should take advantage of powerful storytelling formulas such as comparing opposites (eg, heroes and villains, the tale of two cities), use of repetition and parallel structure, the problem-resolution story device or triumphs of the underdog.

Conclusion

SEIPS 101 and the seven simple SEIPS tools intend to make SEIPS more usable and useful, especially for practitioners and others who may be less acquainted with SEIPS or its uses. The tools are based on our and our colleagues’ experiences but require validation and evaluations of ease of use and usefulness. Such evaluation might assess the extent to which the models and tools allow users to accomplish their tasks effectively, efficiently and satisfactorily, per the standard international definition of usability. Following usability engineering practice, the model’s and tools’ usability can be assessed by expert rating, user self-report and assessing performance during actual use.⁵⁰

Over time, we anticipate consumers of SEIPS 101 and its tools will gain expertise with them, consult the broader literature on SEIPS and related work system models and approaches, report their experiences and adaptations of the tools and partner with other systems thinkers including formally trained human-centred systems engineers to improve efficiency and effectiveness. Ideally, future work will produce additional SEIPS tools and off-the-shelf methods, along with simplifications and tools derived from other theoretical frameworks, for easier and more useful practical application in quality, safety and health.

We thank the many collaborators, contributors, and advocates in the SEIPS community.

Ethics statements

Patient consent for publication

Not required.

Footnote

Contributors RJH and PC jointly conceptualised and wrote this paper.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Carayon P, Schoofs Hundt A, Karsh B-T, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 Suppl 1: i50–8. doi:10.1136/qshc.2005.015842 http://www.ncbi.nlm.nih.gov/pubmed/17142610

² Carayon P, Wooldridge A, Hoonakker P, et al. SEIPS 3.0: Human-centered design of the patient journey for patient safety. Appl Ergon 2020; 84: 103033. doi:10.1016/j.apergo.2019.103033 http://www.ncbi.nlm.nih.gov/pubmed/31987516

³ Holden RJ, Carayon P, Gurses AP, et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56: 1669–86. doi:10.1080/00140139.2013.838643 http://www.ncbi.nlm.nih.gov/pubmed/24088063

⁴ Karsh B-T, Holden RJ, Alper SJ, et al. A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Qual Saf Health Care 2006; 15 Suppl 1: i59–65. doi:10.1136/qshc.2005.015974 http://www.ncbi.nlm.nih.gov/pubmed/17142611

⁵ Carayon P, Wetterneck TB, Rivera-Rodriguez AJ, et al. Human factors systems approach to healthcare quality and patient safety. Appl Ergon 2014; 45: 14–25. doi:10.1016/j.apergo.2013.04.023 http://www.ncbi.nlm.nih.gov/pubmed/23845724

⁶ Werner NE, Ponnala S, Doutcheva N, et al. Human factors/ergonomics work system analysis of patient work: state of the science and future directions. Int J Qual Health Care 2021; 33: 60–71. doi:10.1093/intqhc/mzaa099 http://www.ncbi.nlm.nih.gov/pubmed/33432984

⁷ Waterson P, Robertson MM, Cooke NJ, et al. Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety. Ergonomics 2015; 58: 565–99. doi:10.1080/00140139.2015.1015622 http://www.ncbi.nlm.nih.gov/pubmed/25832121

⁸ Carayon P. Handbook of human factors and ergonomics in patient safety. 2nd edn. Mahwah, NJ: Lawrence Erlbaum, 2012.

⁹ Carayon P. The balance theory and the work system model… twenty years later. International Journal of Human–Computer Interaction 2009; 25: 313–27.

¹⁰ Carayon P. Human factors of complex sociotechnical systems. Appl Ergon 2006; 37: 525–35. doi:10.1016/j.apergo.2006.04.011 http://www.ncbi.nlm.nih.gov/pubmed/16756937

¹¹ Smith MJ, Sainfort-Carayon P. Balance theory of job design for stress reduction. International Journal of Industrial Ergonomics 1989; 4: 67–79.

¹² Donabedian A. The quality of care. How can it be assessed? JAMA 1988; 260: 1743–8.

¹³ Werner NE, Rutkowski R, Graske A, et al. Exploring SEIPS 2.0 as a model for analyzing care transitions across work systems. Appl Ergon 2020; 88: 103141. doi:10.1016/j.apergo.2020.103141 http://www.ncbi.nlm.nih.gov/pubmed/32421635

¹⁴ Zhang J. The nature of external representations in problem solving. Cognitive Science 1997; 21: 179–217.

¹⁵ Zhang J, Norman DA. Representations in distributed cognitive tasks. Cognitive Science 1994; 18: 87–122.

¹⁶ Walker DM, Hefner JL, Sieck CJ, et al. Framework for evaluating and implementing inpatient portals: a multi-stakeholder perspective. J Med Syst 2018; 42: 158. doi:10.1007/s10916-018-1009-3 http://www.ncbi.nlm.nih.gov/pubmed/30014267

¹⁷ Zisberg A, Agmon M, Gur-Yaish N, et al. No one size fits all-the development of a theory-driven intervention to increase in-hospital mobility: the "WALK-FOR" study. BMC Geriatr 2018; 18: 91 https://bmcgeriatr.biomedcentral.com/articles/ doi:10.1186/s12877-018-0778-3 http://www.ncbi.nlm.nih.gov/pubmed/29653507

¹⁸ Carayon P, Gurses AP, Hundt AS. Performance obstacles and facilitators of healthcare providers. In: Korunka C, Hoffmann P, eds. Change and quality in human service work. Munchen. Germany: Hampp Publishers, 2005: 257–76.

¹⁹ Gurses AP, Carayon P. Exploring performance obstacles of intensive care nurses. Appl Ergon 2009; 40: 509–18. doi:10.1016/j.apergo.2008.09.003 http://www.ncbi.nlm.nih.gov/pubmed/18951120

²⁰ Holden RJ. What stands in the way of technology-mediated patient safety improvements?: a study of facilitators and barriers to physicians' use of electronic health records. J Patient Saf 2011; 7: 193–203. doi:10.1097/PTS.0b013e3182388cfa http://www.ncbi.nlm.nih.gov/pubmed/22064624

²¹ Holden RJ, Schubert CC, Eiland EC, et al. Self-Care barriers reported by emergency department patients with acute heart failure: a sociotechnical systems-based approach. Ann Emerg Med 2015; 66: 1–12. doi:10.1016/j.annemergmed.2014.12.031 http://www.ncbi.nlm.nih.gov/pubmed/25616317

²² Acher AW, LeCaire TJ, Hundt AS, et al. Using human factors and systems engineering to evaluate readmission after complex surgery. J Am Coll Surg 2015; 221: 810–20. doi:10.1016/j.jamcollsurg.2015.06.014 http://www.ncbi.nlm.nih.gov/pubmed/26228011

²³ Carayon P, Li Y, Kelly MM, et al. Stimulated recall methodology for assessing work system barriers and facilitators in family-centered rounds in a pediatric hospital. Appl Ergon 2014; 45: 1540–6. doi:10.1016/j.apergo.2014.05.001 http://www.ncbi.nlm.nih.gov/pubmed/24894378

²⁴ Hoonakker PLT, Carayon P, McGuire K, et al. Motivation and job satisfaction of Tele-ICU nurses. J Crit Care 2013; 28: 315–e13. doi:10.1016/j.jcrc.2012.10.001 http://www.ncbi.nlm.nih.gov/pubmed/23159143

²⁵ Salwei ME, Carayon P, Hundt AS, et al. Role network measures to assess healthcare team adaptation to complex situations: the case of venous thromboembolism prophylaxis. Ergonomics 2019; 62: 864–79. doi:10.1080/00140139.2019.1603402 http://www.ncbi.nlm.nih.gov/pubmed/30943873

²⁶ Hundt AS, Carayon P, Yang Y. Role network analysis of team interactions and individual activities: application to VTe prophylaxis. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2017; 61: 896–900.

²⁷ Holden RJ, Daley CN, Mickelson RS, et al. Patient decision-making personas: an application of a patient-centered cognitive task analysis (P-CTA). Appl Ergon 2020; 87: 103107. doi:10.1016/j.apergo.2020.103107 http://www.ncbi.nlm.nih.gov/pubmed/32310109

²⁸ Holden RJ, Kulanthaivel A, Purkayastha S, et al. Know thy eHealth user: development of biopsychosocial personas from a study of older adults with heart failure. Int J Med Inform 2017; 108: 158–67. doi:10.1016/j.ijmedinf.2017.10.006 http://www.ncbi.nlm.nih.gov/pubmed/29132622

²⁹ Holden RJ, Schubert CC, Mickelson RS. The patient work system: an analysis of self-care performance barriers among elderly heart failure patients and their informal caregivers. Appl Ergon 2015; 47: 133–50. doi:10.1016/j.apergo.2014.09.009 http://www.ncbi.nlm.nih.gov/pubmed/25479983

³⁰ Mickelson RS, Holden RJ. Medication adherence: staying within the boundaries of safety. Ergonomics 2018; 61: 82–103. doi:10.1080/00140139.2017.1301574 http://www.ncbi.nlm.nih.gov/pubmed/28394204

³¹ Mickelson RS, Holden RJ. Medication management strategies used by older adults with heart failure: a systems-based analysis. Eur J Cardiovasc Nurs 2018; 17: 418–28. doi:10.1177/1474515117730704 http://www.ncbi.nlm.nih.gov/pubmed/28901787

³² Mickelson RS, Unertl KM, Holden RJ. Medication management: the macrocognitive workflow of older adults with heart failure. JMIR Hum Factors 2016; 3: e27 https://humanfactors.jmir.org/2016/2/e27/ doi:10.2196/humanfactors.6338 http://www.ncbi.nlm.nih.gov/pubmed/27733331

³³ Mickelson RS, Willis M, Holden RJ. Medication-Related cognitive artifacts used by older adults with heart failure. Health Policy Technol 2015; 4: 387–98. doi:10.1016/j.hlpt.2015.08.009 http://www.ncbi.nlm.nih.gov/pubmed/26855882

³⁴ Daley CN, Cornet VP, Toscos TR. Naturalistic decision making in everyday self-care among older adults with heart failure. Journal of Cardiovascular Nursing in press.

³⁵ Ghahari RR, Holden RJ, Flanagan ME. Using cardiac implantable electronic device data to facilitate health decision making: a design study. International Journal of Industrial Ergonomics 2018; 64: 143–54.

³⁶ Adlin T, Pruitt J. The Persona lifecycle: keeping people in mind throughout product design. San Francisco, CA: Morgan Kaufmann, 2010.

³⁷ Heiden SM, Holden RJ, Alder CA, et al. Human factors in mental healthcare: a work system analysis of a community-based program for older adults with depression and dementia. Appl Ergon 2017; 64: 27–40. doi:10.1016/j.apergo.2017.05.002 http://www.ncbi.nlm.nih.gov/pubmed/28610811

³⁸ Johnson CM, Johnston D, Crowle PK. Ehr usability toolkit: a background report on usability and electronic health records (prepared by Westat under contract No. HHSA 290-2009-00023I). AHRQ publication No. 11-0084-EF. Rockville, MD: Agency for Healthcare Research and Quality, 2011.

³⁹ Dul J, Bruder R, Buckle P, et al. A strategy for human factors/ergonomics: developing the discipline and profession. Ergonomics 2012; 55: 377–95. doi:10.1080/00140139.2012.661087 http://www.ncbi.nlm.nih.gov/pubmed/22332611

⁴⁰ Holden RJ, Abebe E. Medication transitions: vulnerable periods of change in need of human factors and ergonomics. Appl Ergon 2021; 90: 103279. doi:10.1016/j.apergo.2020.103279 http://www.ncbi.nlm.nih.gov/pubmed/33049545

⁴¹ Xie A, Carayon P, Cox ED, et al. Application of participatory Ergonomics to the redesign of the family-centred rounds process. Ergonomics 2015; 58: 1726–44. doi:10.1080/00140139.2015.1029534 http://www.ncbi.nlm.nih.gov/pubmed/25777042

⁴² Carayon P, Cartmill R, Hoonakker P. Human factors analysis of workflow in health information technology implementation. In: Carayon P, ed. Handbook of human factors and ergonomics in patient safety. 2nd ed.. Mahwah, NJ: Lawrence Erlbaum, 2012: 507–21.

⁴³ Unertl KM, Novak LL, Johnson KB, et al. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc 2010; 17: 265–73. doi:10.1136/jamia.2010.004333 http://www.ncbi.nlm.nih.gov/pubmed/20442143

⁴⁴ Jun GT, Ward J, Morris Z, et al. Health care process modelling: which method when? Int J Qual Health Care 2009; 21: 214–24. doi:10.1093/intqhc/mzp016 http://www.ncbi.nlm.nih.gov/pubmed/19363123

⁴⁵ Kalbach J. Mapping experiences: a complete guide to creating value through journeys, blueprints, and diagrams. Sebastopol, CA: O'Reilly Media, Inc, 2016.

⁴⁶ Hay GJ, Klonek FE, Parker SK. Diagnosing rare diseases: a sociotechnical approach to the design of complex work systems. Appl Ergon 2020; 86: 103095. doi:10.1016/j.apergo.2020.103095 http://www.ncbi.nlm.nih.gov/pubmed/32342886

⁴⁷ Brown JS, Prusak L, Denning S. Storytelling in organizations: why Storytelling is transforming 21st century organizations and management. New York: Routledge, 2005.

⁴⁸ Holden RJ, Boustani MA, Azar J. Agile innovation to transform healthcare: Innovating in complex adaptive systems is an everyday process, not a light bulb event. BMJ Innovations 2021 https://doi.org/bmjinnov-2020-000574

⁴⁹ Holden RJ, Boustani M. The value of an ‘Agile’ mindset in times of crisis. Modern Healthcare, 2020.

⁵⁰ Nielsen J. Usability engineering. San Francisco, CA: Morgan Kaufmann, 1993.