Managers' use of nursing workforce planning and deployment technologies: protocol for a realist synthesis of implementation and impact

Abstract

Introduction

Nursing staffing levels in hospitals appear to be associated with improved patient outcomes. National guidance indicates that the triangulation of information from workforce planning and deployment technologies (WPTs; eg, the Safer Nursing Care Tool) and ‘local knowledge’ is important for managers to achieve appropriate staffing levels for better patient outcomes. Although WPTs provide managers with predictive information about future staffing requirements, ensuring patient safety and quality care also requires the consideration of information from other sources in real time. Yet little attention has been given to how to support managers to implement WPTs in practice. Given this lack of understanding, this evidence synthesis is designed to address the research question: managers’ use of WPTs and their impacts on nurse staffing and patient care: what works, for whom, how and in what circumstances?

Methods and analysis

To explain how WPTs may work and in what contexts, we will conduct a realist evidence synthesis through sourcing relevant evidence, and consulting with stakeholders about the impacts of WPTs on health and relevant public service fields. The review will be in 4 phases over 18 months. Phase 1: we will construct an initial theoretical framework that provides plausible explanations of what works about WPTs. Phase 2: evidence retrieval, review and synthesis guided by the theoretical framework; phase 3: testing and refining of programme theories, to determine their relevance; phase 4: formulating actionable recommendations about how WPTs should be implemented in clinical practice.

Ethics and dissemination

Ethical approval has been gained from the study's institutional sponsors. Ethical review from the National Health Service (NHS) is not required; however research and development permissions will be obtained. Findings will be disseminated through stakeholder engagement and knowledge mobilisation activities. The synthesis will develop an explanatory programme theory of the implementation and impact of nursing WPTs, and practical guidance for nurse managers.

Trial registration number

CRD42016038132.

Correspondence to Professor Christopher Burton; [email protected]

Strengths and limitations of this study

• The review will address the gap in the evidence about the implementation of nursing workforce planning technologies.

• The realist approach will allow a review of the complexity surrounding the management challenges in workforce planning.

• There will be strong stakeholder engagement to ensure findings have relevance for management practice.

• The relevant literature is diffuse and will require expertise in information science of a realist approach to evidence.

Introduction

Evidence suggests that nursing staffing levels in hospitals are associated with patient outcomes.¹ An important task for nursing managers is to triangulate information from workforce planning and deployment technologies (WPTs) with their ‘local knowledge’ of what is required to achieve appropriate staffing levels for better patient outcomes.² Examples of WPTs include the Safer Nursing Care Tool (SNCT),³ the National Health Service (NHS) England Mental Health Safer Staffing Framework,⁴ the Scottish Workload and Workforce Planning learning toolkit,⁵ the Welsh Adult Acute Nursing Acuity and Dependency Tool,⁶ and the Canadian registered nurse (RN)/Registered Psychiatric Nurse (RPN) usage toolkit.⁷ Evaluation of WPTs has focused predominantly on their development and predictive reliability. However, little knowledge exists about how the implementation of WPTs is effective (or not) in managing the nursing workforce in the real world, and how they work to support safe patient care.⁸ How WPTs are used and interpreted may vary in different organisational contexts. There may also be other more subtle, currently untapped, resources and capacities in the workforce that managers may be using to support the evaluation and deployment of nurse staffing to impact on patient care.

Managing the nursing workforce in the UK generally uses a mix of top–down approaches (eg, benchmarking tools), bottom–up (eg, modelling) or consensus approaches, which are reliant on judgement and intuition for determining nursing requirements.^{5 8} The recent Carter Review for NHS England identified Nursing Care Hours per Patient Day as a calculation method to be adopted by acute hospital services.⁹ In this evidence synthesis, we are interested in the full range of global WPTs that support workforce planning, including those which estimate nursing resources (numbers and/or skill mix); patient needs/dependency/acuity; nursing activity/workload; and the quality and safety of nursing care singly; or more usually in combination. Estimates of nursing care intensity may be based on:

• Patient profiles: descriptions of patient types, associated with needs.

• Critical indicators of care—different levels of care used to classify patients.

• Task-based approaches—for example, nursing information system for change management.

We will use a realist approach to explain the implementation and impacts of WPTs, looking at responses to workforce resource deployment within different contexts. Workforce planning occurs in health settings characterised by fluctuating demands and requirements, and managers play key and challenging roles in complex decision-making around the nursing workforce. While having the potential to inform workforce modelling and establishment setting, WPTs provide a crude prediction of (some) workforce resource requirements. However, they may not reflect real-time resource delivery, which can be eroded by a wide range of factors.⁸ In addition, these approaches may miss the more subtle, human resources in the workforce, such as individual's (including managers') capacities and capabilities, which can be identified and repositioned to ensure the greatest impact on care quality.

Although correlational links have been made between higher nurse staffing level and some patient safety outcomes, such as falls and missed nursing care,¹⁰ little attention has been paid to supporting the implementation of WPTs in clinical practice. The review will fill a gap in the evidence base by focusing on understanding what works for whom, why and in what contexts. We will investigate WPTs that are currently used within different healthcare organisations, to identify and explain what particular features about them are more likely to (or not) promote better quality care for patients. We are interested in how and why WPTs may operate to guide efficient and effective deployment of nursing workforce resources. The findings from this evidence synthesis will equip nursing managers and organisations with guidance to effectively implement WPTs. As far as we are aware, this would be the first evidence synthesis to address this important issue.

Background

NHS organisations have a responsibility to ensure nurse staffing is sufficient for the provision of safe and high-quality care for patients.² Wales is the first country in the UK to legislate a Nurse Staffing Levels Act, but the impact of this is yet to be evaluated.¹¹ WPTs have the potential to ensure safe nurse staffing provision; however, National Institute for Health and Care Excellence (NICE) guidance highlights that insufficient evidence is currently available to show the impact of using particular WPTs.¹² While there has been progress in developing more comprehensive staff mix decision-making tools, there are still gaps in the evidence to show how tools and processes take account of different factors across patient groups, staff groups and organisational systems.¹³ There is insufficient evidence to show links between tools and approaches to assess nurse staffing and patient outcomes.¹² Current evidence focuses on acute care,¹⁴ and most research to date is from North America.¹⁵ Additionally, the uptake and implementation of WPTs appears to vary across organisations.⁵

While there are a considerable number of WPTs available to determine nurse staffing requirements, it is imperative to ensure their accuracy as a basis for resource allocation.¹⁰ It is acknowledged that the use of WPTs must take into account the factors which can influence their effectiveness, including changes in patient acuity,² structural characteristics and organisational systems.¹⁶ The Shelford Group state: “no national workforce tool can incorporate all factors and so combining methods (triangulation) is recommended to arrive at optimal staffing levels. This should include quantitative assessments such as those encapsulated in the SNCT and other more qualitative and professional judgement methods to increase confidence in recommended staffing levels and provide balanced assurance” (p. 3).³ This reinforces that the impact of WPTs will be shaped by their real-time implementation, and through the capabilities and capacities of managers. It highlights managers' leadership role in seeking out and triangulating additional information to appropriately manage nursing resources. Furthermore, the contingencies on which the information that WPTs provide may successfully influence clinical and organisational changes, given the continuing dynamics of healthcare workforce reshaping.¹⁷

Policy guidance indicates that a wide range of factors can mediate the impacts of WPTs, including executive buy-in; staff involvement and transparency in applying the outcomes of technology use and evaluation at the front line.⁸ In this way, the use of WPTs will be dependent on context, may be transformative and potentially change context, so making a simple ‘causal model’ of their action and impact problematic. For example, managers' learning about workforce planning, observations of impacts of different WPTs and improvements over time in the quality of managers' professional judgements around staffing, all may transform context through individual and organisational feedback loops, such as changing organisational-wide planning systems. However, there is another narrative in the literature where professional judgements may become entrenched and uncritical over time.¹⁸

Adopting a realist synthesis approach enables the consideration of additional contextual influences on the impact of workforce planning technologies, and at other levels within the healthcare system. For example, the impact of WPTs may also be variable and contingent on organisational and workforce flexibility; some influences may only emerge through implementation. These influences will be associated with the ‘complex interdependencies between nursing, midwifery and care staffing capacity and capability, and other parts of an organisation's structure and functions’.² This demonstrates how policy and practice around nurse staffing should be integrated with other aspects of organisational practice. Specifically, the use of WPTs should be conceived as part of a much broader and complex system of management practice to ensure quality and patient safety: “safe staffing relies on good management so that budgeted posts are filled, and deployed effectively, and the staff employed are available to work” (p. 5).⁸ Moreover, it challenges those producing and reviewing evidence to understand this system complexity through more nuanced consideration of contextual influences on implementation and impact.

Research question and aims

NHS managers' use of WPTs and their impacts on nurse staffing and patient care: what works, for whom, how and in what circumstances?

The main aims are:

• To identify the different WPTs that could be used to deploy the nursing workforce resource in the NHS, paying attention to the ways in which they are assumed and are observed to work in practice.

• To explore the range of observed impacts of these technologies in different healthcare settings, and for other public services such as social work and policing, paying attention to contextual influences.

• To investigate ways which can help NHS managers identify, deploy and evaluate the nursing workforce planning resource to have greatest impact on direct patient care.

• To generate actionable recommendations for management practice and organisational strategy.

• To contribute to the broader public debate about, and understanding of the nature of the nursing workforce, nursing work, the wider healthcare workforce and the quality of patient care.

Theoretical territory

A realist synthesis has been designed as it is considered to be appropriate approach to answer the synthesis question and aims. Realist synthesis draws on a heterogeneous evidence base to establish whether interventions work or not, how, in what contexts and for whom.^{19 20} It offers the potential to provide practical solutions to, and/or explanations about, challenging problems and issues.

Realist synthesis methodology is located within a critical realist view of causality in the social world.²¹ Within this realist synthesis, the analytical task is to construct, test and refine a programme theory of causal explanations about what works about WPTs. These causal explanations are expressed as plausible hypothesis, or relationships between context, mechanisms and outcomes (C-M-O) to show how certain contexts have triggered mechanisms to generate an observed outcome pattern. The C-M-O framework can draw on mid-range theories to explain how programmes work, or not, through examining patterns (demiregularities) of outcomes for particular contexts.¹⁹ The approach is based on negotiation between stakeholders and reviewers, so stakeholder engagement is high¹⁹ and contributes towards the formulation and refinement of programme theories.^{20 22} Conventional, Cochrane-style systematic reviews tend to focus on evidence of effectiveness with narrowly focused questions; in contrast, the realist approach has the potential to unpack complex, contextually contingent issues, such as in the case of this proposal about WPTs.

Realist syntheses are theory-driven. The synthesis is designed to will test a programme theory, capturing the complexity of interactions to offer an explanatory account of how WPTs work. An initial theoretical framework will be developed, informed by a scoping review of the evidence and consultation with stakeholders. The framework will provide a provisional (hypothetical) explanation of what works and the impact of WPTs by investigating literature and evidence from separate but interlinked disciplines, around two theory areas: the elements of workforce planning themselves and their implementation (see table 1).

Table 1

Theory areas

Elements of workforce planningImplementation

The identification of patient needs and acuity23 24 The nature of nursing work25 Scope of workforce planning (eg, reflecting integrated care and skill mix changes)26 27 Contracting and rostering practices28–31 Deployment, skill mix and nursing workload tools8 Strategic management and human resources for health32 33	Technology adoption34 Professional decision-making and judgement35 Organisational and other contextual influences affecting the implementation of learning and practices36–38 Organisational learning and knowledge management39 Implementation and knowledge mobilisation40–42

We are interested in identifying the full range of potential WPTs impacts, and which may extend beyond healthcare. These impacts may also relate to evidence about workforce (eg, staff satisfaction) and organisation theories (eg, organisational learning). Different impacts from WPTs will be noted on a continuum, ranging from conceptual, instrumental or direct from recognition, knowledge and understanding, attitudes and insights, to changes in managers' and organisational behaviour.⁴³

Methods

Through following recognised reporting frameworks,⁴⁴ and the stages of realist synthesis,^{19 20} the synthesis will be conducted in four phases over 18 months

• Programme theory development.

• Evidence search, retrieval, review and extraction.

• Programme theory testing and refinement through evidence synthesis.

• Development of actionable recommendations.

While these phases are described sequentially, in practice there is considerable overlap between them. However, stakeholder engagement is embedded throughout. The study advisory group will guide on policy and organisational engagement. Members of the group will include senior representatives from health, social care and public services with high-level experience of workforce planning design and delivery. Additionally, patient and public involvement (PPI) representatives are recruited to the project team to inform programme theory development, interpretation and dissemination of findings. Throughout the study's lifespan, generated knowledge will be mobilised through the use of social media, engagement and dissemination activities.

Phase 1: programme theory development

We will construct the review's initial programme theory from the underpinning evidence in consultation with stakeholders. To develop an understanding of the complexity of the contexts in which systems and technologies are used, we will draw on soft systems thinking to structure two co-production workshops with nursing managers and other stakeholders.⁴⁵ We will also plan to conduct interviews with a purposive sample of nursing managers to build on the information from the co-production workshops, and ensure we have captured variations in workforce planning systems across organisational settings and health services. The resulting initial programme theory will provide an initial explanation of the complexity of using WPTs for evaluation within the review.

Phase 2: evidence retrieval, data extraction and evidence synthesis

In phase 2, we will search for relevant evidence related to nursing WPTs to test and refine the programme theory. The process will involve screening evidence for relevance, data extraction and charting. The realist approach enables emerging findings to be tested across one body of literature to another, to determine if other literatures offer transferable understanding on context and mechanisms, which are transferable. We will target evidence specific to the nursing workforce in the first instance, across hospital, community and third-sector care in the context of UK and comparable health systems. This will be complemented by further searches to test the impacts of WPTs in related service fields, for example, social care and policing, where there may be comparable workforce planning requirements.

Search strategy

A realist approach offers the opportunity to explore an eclectic range of the evidence.¹⁹ To ensure relevance, our search will be limited from 1983 to current date. This year saw the commission of the NHS Management Inquiry to evaluate methods of estimating staffing levels, and the classification of workload analysis approaches by the Operational Research Service of the then Department of Health and Social Security.¹⁶ We intend to include material indexed in the major health and related databases, including the following: MEDLINE, CINAHL, HMIC, PubMed and Cochrane library. Keywords will be developed from previous systematic reviews and adapted for each information source. The search terms of workforce planning systems and technologies will be constructed from a mix of database-specific ‘keywords’. Additional search terms will enable concentration on issues of usage, implementation and impact. The search references will be augmented by searches for generic quality improvement and organisational development programmes which make specific reference to workforce planning. We will also conduct internet searches for grey literature, such as workforce planning project reports related to national and local initiatives and seek evaluative information on these initiatives. We will use snowballing techniques and cluster searching⁴⁶ and draw on the expertise of the advisory group to ensure that evidence of relevance will not be missed.

Inclusion and exclusion criteria

Our search strategy will be purposive to test the programme theory and inform C-M-O refinement. We will use a systematic process to determining relevance developed in a current realist synthesis.⁴⁷ Consistent with Pawson's¹⁹ suggestion, the test for inclusion will be:

• Linkage with programme theory and explanatory potential;

• Discernible ‘nuggets’ of evidence within the source material;

• Evidence of trustworthiness.

We will include reports of WPTs, including workforce planning; workforce measurement; workforce management; patient acuity; patient dependency; staffing ratios; professional judgement and skill mix. We will also search for evidence on settings, recognising the shifting patterns of healthcare and the importance of enabling patient flow and quality across systems of care.

In a realist synthesis, evidence is only excluded if it does not relate to, or inform the development of the programme theory; however, in this review, we will not include evidence that has limited transferability to the NHS, such as nursing workforce issues within low-income countries. We will only include evidence generated from different international contexts in comparable health systems. Discrepancies in opinions on the relevance of evidence will be resolved through discussion among the project team.

Data extraction

In a realist evidence synthesis, bespoke data extraction forms are developed to guide the decision-making process.¹⁹ Based on the programme theory of WPTs, we will develop a bespoke extraction form to interrogate the theories and extract data only if the evidence meets the test of relevance for the programme theory. A selection of included data will be validated by a second member of the team.

Synthesis

Relationships between mechanisms, contexts and outcomes will be analysed from the extracted information. We will follow an approach to synthesis formulated from our previous experience of realist synthesis⁴⁸ and which builds on Pawson's¹⁹ suggestions and the principles of realist enquiry. This involves organising extracted data into evidence tables to represent the different bodies of literature. Using abduction and retroduction⁴⁹ across the evidence tables, we aim to reconceptualise WPTs from different angles to identify underlying structures and emerging demiregularities (patterns) around plausible C-M-Os, seeking confirming and disconfirming evidence. These demiregularities will be linked to develop programme theory which provides an explanation of the implementation, usage and impacts of WPTs.

The resultant hypotheses act as synthesised statements of findings around which a narrative can be developed, summarising the nature of the C-M-O links, and the specific characteristics of the evidence underpinning them. Outputs will include a comprehensive evidence base relevant to WPTs to support a set of hypotheses to be refined in phase 3.

Phase 3: testing and refining the programme theory

To refine the programme theory, with accompanying evidence-based narrative, we will conduct up to 10 semistructured audio-recorded telephone interviews with a purposive sample of NHS nursing workforce and other managers. This will provide different perspectives relevant to the review question, including different national contexts, and service settings. An interview schedule will be developed based on the findings from the synthesis process to elicit stakeholder's views on their resonance, and ensure trustworthiness of the resultant programme theory. Additionally, the PPI representatives will be asked to assess the relevance of the mechanism-context-outcome threads (ie, hypotheses) from a service user perspective. This activity will be undertaken on an ongoing basis by view of their involvement on the project team and the advisory group.

Phase 4: actionable recommendations

Within this phase, we will engage with the advisory group including PPI members, to develop a set of actionable recommendations and an evidence informed framework of what works for whom, and in what context with the implementation and use of WPTs. We will achieve this via meetings and teleconferences, and via a knowledge mobilisation event with a group of stakeholders to ensure the recommendations we develop are both pertinent and actionable.

Ethical issues

The interviews conducted as part of phases 1 and 3 will be undertaken with the staff. Ethical review from the NHS is not required; however, local research and development permissions will be gained before access to site.

Project outputs

Using our synthesis findings, we will recommend a series of improvement resources and support for managers in this aspect of their work, including:

• A final research report, using vignettes of different examples/case studies to illustrate findings, and a framework for managers around workforce planning for skills development and learning.

• An executive summary of the final report for briefing managers.

• A lay summary of the final report, as a separate report for the public.

• A benchmarking or quality assurance framework for workforce planning interventions and their implementation.

• Two open-access publications: (1) a review protocol, and (2) a findings paper that sets out an implementation plan of nursing workforce planning systems and technologies across all care sectors.

• A conference presentation at a UK national conference.

• A YouTube animation of the main findings, including a discussion with stakeholders about their relevant to practice and policy.

• Open-access articles in professional and academic journals.

The project website and twitter account will provide a real-time report of progress.

Specifically, the study will provide:

• 1. A description of the nursing WPTs that have been used and evaluated for improving the quality of nursing care. This will explain how they work and their intended and unintended outcomes, therefore, facilitating managers and policymakers to gain an understanding of the range of technologies available, and the key assumptions on how they are supposed to work.

• 2. An explanatory account of the impact of contextual influences on the effective use of technologies in ensuring efficiency in the management of the nursing resource. The influence of context is critical to the outcomes programmes achieve. The synthesis will provide managers and policymakers with the detailed information required to address local contextual issues.

• 3. An evidence-informed framework addressing what works for whom and in what context in relation to WPTs for improving the quality of nursing care. This could be used by organisations to improve this aspect of the management role through facilitation of the identification of suitable professional development strategies to improve implementation and impact. Our stakeholder engagement means that managers will be able to co-produce these development strategies with the project team.

Discussion

Nursing input is essential for high-quality patient care.⁵⁰ This synthesis is important for patients, families, nursing managers and organisations as the association between nurse staffing levels and patient outcomes is acknowledged as a political imperative. Recent high profile reports which focus on the association between nurse staffing and patient safety outcomes, and which associate insufficient nurse staffing numbers with compromised care make this issue an increasingly public imperative. Through this review, we will answer questions that have practical relevance to service delivery and decision makers, including identifying the core ingredients of WPTs, how they should be implemented and what should be the expected impacts on organisational efficiency, care standards and quality.

Our findings have the potential to improve patient outcomes, although we recognise that to date, the links between WPTs use and important patient outcomes has not been easy to explain. For example, there are gaps in the current evidence base that explains the mechanisms by which staffing levels directly impact on patient outcomes.⁵¹ There is limited information on which patient safety outcomes are appropriate to consider (and the credibility of case ascertainment); poor attention to risk adjustment; and little attention is generally paid to organisational factors which may mediate the link between the numbers of nurses and high-quality care. It is therefore important that the synthesis is able to connect and provide clarity between these factors to provide information on which WPTs may work better in different contexts and why. Our work will be of direct benefit to health and social care services in providing a resource to inform development programmes for nursing managers to address the implementation of nursing WPTs.

Attention to implementation and the contextual influences on the impacts of WPTs will mean that barriers and enablers can be identified, and subsequently used to enhance managers' professional judgements and decision-making processes. The development of theoretically informed statements about ‘what works’ in workforce planning within different contexts will increase the transferability of research outputs; the findings from the review will likely be of interest beyond health.

The authors would like to acknowledge Mrs Denise Fisher for her input into the development of the programme theory.

Footnote

Twitter Follow Jo Rycroft-Malone at @jorycroftmalone, Lynne Williams at @lynneolyn, Christopher Burton at @chrisburton5 and ResINPUT at @ResINPUT

Contributors All authors provided substantial contributions to protocol conception and design. CB, JR-M, LW and SD formulated the initial draft. CB, JR-M, LW, SD, AM, BH, A-MR and AJ revised the manuscript critically for intellectual content, and agreed final approval of the version to be published.

Funding This work is supported by the National Institute Health Research (NIHR) Grant no. 14/194/20. This project is commissioned by the NIHR Health Services and Delivery Research programme.

Disclaimer The views and opinions here are those of the authors and do not necessarily reflect those of the Health Service and Delivery Research Programme, NIHR, National Health Service (NHS) or Department of Health.

Competing interests None declared.

Ethics approval HMS Ethics Committee, Bangor University, Bangor UK.

Provenance and peer review Not commissioned; peer reviewed for ethical and funding approval prior to submission.

Data sharing statement A protocol was formulated during the grant application process and submitted to the National Institute for Health Research: Health Services and Delivery Research Programme.

References

¹ Griffiths P, Ball J, Murrells T, et al. Registered nurse, healthcare support worked, medical staffing levels and mortality in English hospital trusts a cross-sectional study. BMJ Open 2016; 6: e008751. doi:10.1136/bmjopen-2015-008751

² Chief Nursing Officer for England & National Quality Board. How to ensure the right people, with the right skills, are in the right place at the right time: a guide to nursing, midwifery and care staffing capacity and capability. 2013. https://www.england.nhs.uk/wp-content/uploads/2013/11/nqb-how-to-guid.pdf

³ The Shelford Group. Safer Nursing Care Tool. Implementation Resource Pack. The Shelford Group, 2013. http://www.nhsiq.nhs.uk/media/2760784/130719_shelford_safer_nursing_final.pdf

⁴ NHS England. Mental health staffing: a practical guide. Part of Compassion in Practice Programme. West Midlands: Health Education, 2015.

⁵ NHS Education for Scotland. Nursing and midwifery workload and workforce planning. Learning toolkit. Edinburgh: The Scottish Government, 2013.

⁶ Welsh Government. Fundamentals of care system. Adult acute nursing acuity and dependency tool. User guide. Welsh Government, 2015.

⁷ HHR Demonstration Project. RN/RPN utilization toolkit project. Ontario: Healthforce Ontario, 2009.

⁸ Royal College of Nursing. Guidance on safe nurse staffing levels in the UK. London: Royal College of Nursing, 2010.

⁹ Lord Carter of Coles. Operational productivity and performance in English NHS acute hospitals: unwarranted variations. An independent report for the Department of Health, 2016.

¹⁰ Griffiths P, Ball J, Drennan J, et al. The association between patient safety outcomes and nurse/healthcare assistant skill mix and staffing levels and factors that may influence staffing requirements. Centre for Innovation and Leadership in Health Sciences, University of Southampton, 2014.

¹¹ Nurse Staffing Levels (Wales) Act (Anaw 5). Great Britain: National Assembly for Wales, 2016. http://legislation.data.gov.uk/anaw/2016/5/body/data.htm?wrap=true (accessed 22 Aug 2016).

¹² National Institute for Health and Care Excellence. Safe staffing for nursing in adult inpatient wards in acute hospitals. National Institute for Health and Care Excellence. 2014. https://www.nice.org.uk/guidance/sg1

¹³ Harris A, McGillis-Hall L. Evidence to inform staff-mix decision making: a focused literature review. Ontario: Canadian Nurses Association, 2012.

¹⁴ Royal College of Nursing, Safe Staffing for older people's wards. RCN full report and recommendations. London: Royal College of Nursing, 2012.

¹⁵ West E, Barron DN, Harrison D, et al. Nurse staffing, medical staffing and mortality in intensive care: an observational study. Int J Nurs Stud 2014; 51: 781–94. doi:10.1016/j.ijnurstu.2014.02.007

¹⁶ Scott C. Setting safe nurse staffing levels. An exploration of the issues. London: Royal College of Nursing, 2003.

¹⁷ Imison C, Castle-Clarke S, Watson R. Reshaping the workforce to deliver the care patients need. Nuffield Trust Research Report. 2016. http://www.nuffieldtrust.org.uk/node/4651

¹⁸ Proctor S. Subjectivity and objectivity in the measurement of nursing workload. J Clin Nurs 1992; 1: 123–9. doi:10.1111/j.1365-2702.1992.tb00082.x

¹⁹ Pawson R. Evidence-based policy: a realist perspective. London: Sage, 2006.

²⁰ Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implement Sci 2012; 7: 33. doi:10.1186/1748-5908-7-33

²¹ Wand T, White K, Patching J. Applying a realist(ic) framework to the evaluation of a new model of emergency department based mental health nursing practice. Nurs Inq 2010; 17: 231–9. doi:10.1111/j.1440-1800.2010.00488.x

²² Pawson R, Tilley N. Realist evaluation. London: Sage, 1997.

²³ Malloch K, Conovaloff A. Patient classification systems, part 1: the third generation. J Nurs Adm 1999; 29: 49–56. doi:10.1097/00005110-199907000-00013

²⁴ Van Slyck A, Johnson KR. Using patient acuity data to manage patient care outcomes and patient care costs. Outcomes Manag Nurs Pract 2001; 5: 36–40.

²⁵ Allen DA. The invisible work of nurses: hospitals, organisation and healthcare. New York: Routledge, 2014.

²⁶ Villarreal MC, Keskinocak P. Staff planning for operating rooms with different surgical services lines. Health Care Manag Sci 2016; 19: 144–69. doi:10.1007/s10729-014-9307-x

²⁷ Sibbald B, Shen J, McBride A. Changing the skill-mix of the healthcare workforce. J Health Serv Res Policy 2004; 9 (Suppl 1): 28–38. doi:10.1258/135581904322724112

²⁸ Burke EK, De Causmaecker P, Vanden Berghe G, et al. The state of the art of nurse rostering. J Scheduling 2004; 7: 441–99. doi:10.1023/B:JOSH.0000046076.75950.0b

²⁹ Duffield C, Diers D, O'Brien-Pallas L, et al. Nursing staffing, nursing workload, the work environment and patient outcomes. Appl Nurs Res 2011; 24: 244–55. doi:10.1016/j.apnr.2009.12.004

³⁰ Twigg DE, Duffield CM, Bremner A, et al. Impact of skill mix variations on patient outcomes following implementation of nursing hours per patient day staffing: a retrospective study. J Adv Nurs 2012; 68: 2710–18. doi:10.1111/j.1365-2648.2012.05971.x

³¹ Blay N, Duffield CM, Gallagher R, et al. Methodological integrative review of the work sampling technique used in nursing workload research. J Adv Nurs 2014; 70: 2434–49. doi:10.1111/jan.12466

³² Boxall P, Purcell J. Strategy and human resource management. 3rd edn. Basingstoke: Palgrave Macmillan, 2011.

³³ World Health Organization. Working together for health: the World Health Report. Geneva: World Health Organization, 2006.

³⁴ Rogers EM. Diffusion of innovations. New York: Free Press, 2003.

³⁵ Thompson C, Dowding D. Clinical decision making and judgement in nursing. London: Churchill Livingstone, 2009.

³⁶ Easterby-Smith M. Disciplines of organizational learning: contributions and critiques. Hum Relations 1997; 50: 1085–113.

³⁷ Raelin JA. Work-based learning in practice. J Workplace Learn 1998; 10: 280–3. doi:10.1108/13665629810236183

³⁸ Dewing J. Becoming and being active learners and creating active learning workplaces: the value of active learning. In: McCormack B, Manley K, Wilson V. eds. International practice development in nursing and healthcare. Oxford: Oxford Blackwell, 2008: 273–94.

³⁹ French B, Thomas LH, Baker P, et al. What can management theories offer evidence-based practice? A comparative analysis of measurement tools for organisational context. Implement Sci 2009; 4: 28. doi:10.1186/1748-5908-4-28

⁴⁰ Rycroft-Malone J, Harvey G, Seers K, et al. An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs 2004; 13: 913–24. doi:10.1111/j.1365-2702.2004.01007.x

⁴¹ Ferlie E, Crilly T, Jashapara A, et al. Knowledge mobilization in healthcare organizations: a view from the resource-based view of the firm. Int J Health Policy Manag 2015; 4: 127–30. doi:10.15171/ijhpm.2015.35

⁴² Crilly T, Jashapara A, Trenholm S, et al. Knowledge mobilisation in healthcare organisations: synthesising evidence and theory using perspectives of organisational form, resource based view of the firm and critical theory. National Institute of Health Research Health Services and Delivery Research programme, 2013.

⁴³ Nutley S, Walters I, Davies HTO. Using evidence. How research can inform public services. Bristol: Policy Press, 2007.

⁴⁴ Wong G, Greenhalgh T, Westhorp G, et al. RAMESES publication standards: realist synthesis. J Adv Nurs 2013; 69: 1005–22. doi:10.1111/jan.12095

⁴⁵ Checkland P. Systems thinking, systems practice. Chichester: Wiley, 1999.

⁴⁶ Booth A, Harris J, Croot E, et al. Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex interventions: case study (CLUSTER). BMC Med Res Methodol 2013; 13: 118. doi:10.1186/1471-2288-13-118

⁴⁷ Burton C, Rycroft-Malone J, Hall B, et al. From rhetoric to reality: stakeholders’ involvement in realist synthesis. The 1st International Conference on Realist Approaches to Evaluation and Synthesis: Successes, Challenges, and the Road Ahead. Liverpool. 2014. http://programme.exordo.com/cares2014/delegates/presentation/14/

⁴⁸ Rycroft-Malone J, Burton C, Hall B, et al. Improving skills and care standards in the support workforce for older people: a realist review. BMJ Open 2014; 4: e005356. doi:10.1136/bmjopen-2014-005356

⁴⁹ Meyer SB, Lunnay B. The application of abductive and retroductive inference for the design and analysis of theory-driven sociological research. Sociological Research Online, 2012. http://www.socresonline.org.uk/18/1/12.html

⁵⁰ Kane R, Shamlyian T, Mueller C. et al. Nurse staffing and quality of patient care. Agency for healthcare research and quality. Evidence Report/Technology Assessment. No 151, 2007.

⁵¹ Ball JE, Murrells T, Rafferty AM, et al. “Care left undone” during nursing shifts: associations with workload and perceived quality of care. BMJ Qual Saf 2014; 23: 116–25. doi:10.1136/bmjqs-2012-001767