Correspondence to Dr Rachel Brimelow; [email protected]
STRENGTHS AND LIMITATIONS OF THIS STUDY
The first stage of this study will use a concept mapping and codesign approach to develop the Mental Health benchmarking Industry Tool for residential aged Care (MHICare Tool) involving key stakeholders: residential aged care (RAC) staff, RAC residents with and without dementia and family carers of residents, thus ensuring the MHICare Tool represents stakeholders’ expectations, needs and preferences.
In the second stage of this study, the Delphi process will involve eliciting feedback and reaching consensus on the proposed MHICare Tool from an invited expert panel including practical/clinical experts from RAC, academic and research experts and industry and aged care consumer representatives.
The sorting and rating activities within the Group Concept Mapping (GCM) process may be challenging for some participants, especially those living with dementia. We will use adapted GCM methods where appropriate.
High attrition rates through the three rounds of the Delphi process may impact the quality of the MHICare Tool. Mitigation strategies include maintaining communications with the experts and providing them with regular timeline reminders.
Introduction
In Australia, permanent long-term care is provided in residential aged care facilities (RACFs) to those who are assessed as no longer being able to remain living in their own homes in the community. RACFs in Australia are government funded and provide appropriate accommodation, personal care services, nursing care and equipment as required to meet the needs of its residents.1 Most service providers in Australia are not-for-profit organisations (60%), for example, state-run, community, religious or charitable organisations.2
Mental health conditions are prevalent within RACFs and comprise the full breadth of conditions experienced by younger adults as well as changed behaviours and psychological symptoms associated with dementia.3 4 The identification and ongoing treatment of these conditions is contextually different from those experienced by people within the community setting.3 4 Access to adequate psychiatric consultation and educational services for changed behaviours and psychological symptoms are significantly limited within RACFs and access to quality mental health services is a prominent concern among aged care administrators, clinicians, advocates and policymakers,3 as well as residents and their families/care partners.5 Access to private psychological services such as psychologists through government-funded agencies is restricted within Australia6 and professional care staff exhibit ambivalent attitudes surrounding psychotherapy for older adults.7 Services within RACFs are typically provided by professionals who are not specifically trained and employed in mental healthcare, with nurses and care staff receiving very little or no geropsychology training.8 However, care planning by aged care staff must account for both medical and behavioural health comorbidities.9
Improvements to the management of mental health of residents in residential aged care (RAC) are now pivotal. In Australia, for example, 57% of RAC residents have at least one mental health disorder, 46% present with depression,10 over two-thirds (68%) have been identified as having a cognitive impairment and over half have a diagnosis of dementia.11 The new Aged Care Quality Standards in Australia were introduced in 2019 with greater focus on resident outcomes and minimising the use of physical and chemical restraints, which are thought to be used with up to 40% of RACF residents in Australia.12 13 Accumulating evidence indicates that non-pharmacological approaches can improve changed behaviours, psychological symptoms and other resident mental health outcomes,13–15 are cost-effective14 and are preferred approaches for people living with dementia.5 Factors that positively influence the mental health of residents, both with and without dementia, include organisational policy and procedures, the care environment, staff attitudes, skills, knowledge and resources, as well as the enactment of commonly cited clinical care and psychosocial interventions that focus on person-centred care.14 15 However, significant difficulties have been observed in sustaining implementation and change in practice beyond the intervention period.14 Better system governance is crucial to the reform of mental healthcare practices within aged care to improve outcomes for the high proportion of residents experiencing changed behaviours and psychological symptoms. Effective governance, with clear performance management systems, is required to steer the aged care sector towards long-term policy outcomes, including a culture of ongoing performance monitoring, transparency and accountability that ensures issues are identified and addressed.16 Current accreditation standards of the Australian RAC industry do not allow ongoing quality monitoring of mental healthcare practices at a facility level, and there is no RAC-specific standardised mental health benchmarking tool at an industry level. Not surprisingly, the Australian Royal Commission into Aged Care Quality and Safety identified a lack of appropriate mental health evaluation and treatment practices as a significant unmet need in RAC industry.12
The balanced scorecard (BSC), a strategic planning and management framework first proposed by Kaplan and Norton in 1992,17 has been adapted to many industries, including healthcare.18 The BSC framework provides information on areas of strategic importance to assess current system performance and guide future planning.19 20 The BSC approach works by analysing information on a balanced set of perspectives and indicators derived from the strategic vision of an organisation or industry.21 A review of industry-adopted BSCs within mental health services reported client involvement, client satisfaction and staffing levels among other identified domains.22 The main barriers for successful implementation of such an approach in RACFs included a lack of clear mission strategy, determining appropriate indicators and gaining and maintaining commitment.22
Aim of this study
In response to the identified gap in mental health quality monitoring within an Australian aged care context, we aim to codesign and develop a Mental Health Benchmarking Industry Tool for Residential Aged Care (MHICare Tool) to help mitigate deficits and maximise opportunities for mental health practices at a facility and industry level. The MHICare Tool will adopt the BSC approach and be designed and developed with key stakeholders (residents, family, staff and industry) involved in decision-making from the very beginning, to ensure that the strategic mission and included indicators represent both community and industry needs.
Methods
This protocol focuses on the quality indicator content codesign and development of the MHICare Tool through a shared understanding of resident mental health contributory factors and outcomes that may be amenable to change.
Study design
A two-stage sequential and mixed methods codesign process will be used to develop appropriate quality indicators within the MHICare Tool:
Stage 1: Qualitative focus groups and/or interviews using an adapted concept mapping approach (aCMA). This stage involves engaging aged care residents, family members of residents and RACF employee stakeholders in a brainstorming and CMA to discern key factors of resident mental health that are deemed both highly important and highly changeable (ie, improvements can be made).
Stage 2: Delphi procedure that involves a panel of industry, clinical and academic experts to reach consensus on those identified quality indicators most feasible to collect and providing greatest benefit for improving RACF mental healthcare practices and resident outcomes.
Stage 1: codesign of the MHICare Tool using an aCMA
Engaging residents, family members of residents and RACF employees in identifying what is important to them regarding resident mental health is imperative to ensure that the MHICare Tool represents stakeholders’ expectations, needs and preferences.
Specifically, we aim to address the following research question:
What do residents with and without dementia, family members and RAC staff identify as key areas of need to be addressed and measured within a BSC approach to improve mental health outcomes for residents within RAC?
Participants and setting
The setting for this study is Australian RACFs. Eligible participants include the following groups. We aim to recruit 6–10 participants in each group.
RACF at the management, nurse unit manager level and relevant corporate staff as identified by RACFs, as well as professional care staff, including personal carers/assistant nurses, leisure, and lifestyle coordinators and similar roles.
RACF residents with mild to moderate cognitive impairment or dementia.
RACF residents without cognitive impairment or dementia.
Family members and informal care partners of RACF residents.
Participants will need to be able to speak and understand spoken English to participate in discussions and concept mapping (CM) activities. Written informed consent will be obtained from either the resident or from their trusted proxy if unable to consent due to significant cognitive or physical impairment. Residents must be able to understand what taking part involves (ie, be able to communicate back to the researcher what taking part involves after sensitive discussion with a trained and experienced researcher). The residents have the option to be accompanied by a support person of their choice. Any support person, family or staff member will also be required to complete a consent form.
Recruitment
We will use several recruitment strategies including the Australian ‘StepUp for Dementia Research’ online recruitment tool. This is a self-registration service that enables volunteers with memory problems, mild cognitive impairment or dementia, professional and informal carers and healthy volunteers to register their interest in taking part in research. StepUp for Dementia Research is funded by the Commonwealth Department of Health and delivered by the University of Sydney. StepUp for Dementia Research is not a research project but has been approved by the Human Research Ethics Committee of the University of Sydney (project number: 2018/680).
We will also contact RACFs across Australia to aid with recruitment on confirmation with the organisation of their participation and relevant industry governance.
The research team will send out information and consent forms as appropriate. Participants will be encouraged to discuss participation with family, friends and/or professionals as appropriate before signing the consent form and may withdraw from the study at any time, without care or relationships being affected with clinicians or staff at the aged care facility or university.
Adapted CM methodology
CM methodology (CMA) is a structured consensus-building approach, which consists of one to two rounds of data structuring and is appropriate for consumer participants.23 Sorting, labelling and rating of discreet statements generated through either qualitative interviews or focus groups are aggregated and transformed into quantitative data using rapid multivariate statistical methods and hierarchical cluster analysis to generate concept maps. The effect of which is that consensus emerges organically from the data.23–25 An aCMA will involve six stages.24–26 The methodology will be used to enable stakeholders, including residents with mild to moderate cognitive impairment and dementia, to identify RACF mental health issues and care practices they find relevant and important and will be modified accordingly to adjust for the complex problem-solving skills needed in traditional models of CM. Details of the modified six CM stages are as follows.
Preparation
The preparation stage involves introducing participants to the topic through discussions conducted either face-to-face, via telephone, or via videoconferencing or in small groups, depending on participant preference, prevailing COVID restrictions, location, and availability.
Participants will be encouraged to reflect on: (1) impact of mental health issues, (2) current management of mental health in RACF, (3) practices to be improved, (4) resources and (5) RACF staff training and knowledge (see discussion guide in online supplemental appendix 1). This stage will be used to establish a relationship between the participants and facilitator, which is key for ongoing engagement. Brainstorming discussions will take no more than 1 hour to avoid fatigue. The facilitator will be a researcher who is experienced in conducting qualitative research with older people, RAC staff, people with dementia and family/care partners. A moderator will take notes on interactions and any non-verbal responses in group discussions. Reflective notes will also be made by facilitators regarding what worked well or not, and how solutions may be developed for future sessions. The facilitator will ensure that everyone will have the opportunity to voice their opinion in a respectful manner. One-on-one and group discussions will be audio recorded (with consent) and transcribed verbatim for qualitative data analysis.
Statement generation
In this aCMA methodology, qualitative thematic analysis of the transcribed focus groups/interview data will be conducted using NVivo Release 1.7.27 NVivo software enables electronic data management, flexible approaches to coding, as well as supporting numerical counts.27 28 An inductive approach will be undertaken to derive common themes relevant to the development of the MHICare Tool. Units of meaning within texts and keywords will be grouped together under subcategories, and similar subcategories will then be grouped to identify major themes.
Themes/codes derived from the qualitative analysis will be used to generate statements for CM. Each discrete statement must be focused on one main topic/theme and cannot be worded as a question. Following this session, all discrete statements will be compiled by the research team into a single list, with any duplicate statements removed.
Sorting
The structuring stage involves participants sorting discrete statements into conceptually similar groups. The process will allow each participant to create their own categories based on how similar they believe the statements are.25 Each statement can only be placed into one category and participants can have as many separate categories as they feel necessary. A list of all statements will be provided to participants prior to scheduled interview in preparation and will be presented to the participant individually to be completed with guided facilitation, either online or face to face. The research team identifies that some residents will face challenges with this task. To combat potential difficulties for residents, adaptations will be made including providing short, plain language statements, use of sticky notes or cards, contrasting colours and visuals to aid in creating and sorting groups. This will be adapted based on participant feedback. Researchers will also sort the statements into categories as a backup mechanism.
Rating
Following this, participants will be asked to rate each statement on how important the statement is to them and how changeable they perceived each statement to be on a 4-point Likert scale (0=‘not important’ to 4=‘very important’, and from 0=‘hard to change’ to 4=‘easy to change’).29 30 RACF residents and informal care partners will also be provided with the Likert scale in visual analogue scale format to aid comprehension. For RACF residents, rating will take place in a separate session to combat fatigue and cognitive load. For RACF professional staff and informal care partner codesign members, rating will take place during the same session as the sorting dependent on feedback surrounding time burden and cognitive load.
Representation and interpretation
These data will be analysed using Group Wisdom, specialist CM data collection and analysis software.31 After sorting is completed, multidimensional scaling will be used to create a two-dimensional point map of sorted statements to visualise similarity between participant-generated themes. A cluster map will reveal distinct boundaries of themes (clusters).24 25 32 33 Pattern matching, the process whereby ratings of importance and changeability are depicted by comparing the average of all statements within each cluster, will then be carried out. Following this, Go-Zones (figure 1) will be generated. This produces a visual plot with all individual ideas within a cluster on resident ratings of changeability and importance.24 31
Utilisation
The Go-Zone generated from this CM exercise will be directly used to inform the draft performance indicators of the MHICare Tool to improve the mental health outcomes of residents in RACFs (figure 1).
Adaptability and flexibility of the research project is central to successes. The codesign process will be flexible and will adapt iteratively in response to identified barriers and enablers to ensure active engagement and contribution from participants in the CM process. Participation in CM may comprise several different methods or combinations of methods for brainstorming, sorting and rating procedures. Adaptations to the CM stage modalities also include combinations of in-person, online, group and individual discussions to account for participant preferences.34
Stage 2: Delphi study
A Delphi study will be conducted to obtain industry-wide and expert consensus on the measures identified in the ‘Go-Zone’ to be included in the MHICare Tool.
Specifically, we aim to address the following research question:
What is the consensus of expert views (RAC industry and aged care representative groups, academic, clinical) on the performance indicators to be included in the MHICare Tool for mental health in RAC?
Delphi procedure
The Delphi procedure is an iterative process, designed to reach consensus through eliciting feedback from an expert panel. It is a controlled feedback process that has several advantages including the limiting of group effects and cost-effectiveness.35 A minimum of two rounds will be required, with three rounds usually sufficient to reach 75% agreement for consensus.35 Round 1 will contain a summary of the focus group/interview and CM data as well as a selection of qualitative and quantitative questions including issues surrounding practical implementation. Round 2 will contain a mix of qualitative and quantitative questions to respond to the draft indicators, and round 3 will consist of quantitative ratings of the MHICare Tool items to establish consensus. The survey for each round will be developed in response to previous results. Responses will be measured on a 5-point Likert scale (1 indicating strong lack of agreement and 5 indicating excellent agreement) and measuring importance and feasibility. Completion of surveys will be online using online survey platform, Qualtrics (2023 Qualtrics), hosted at The University of Queensland. Qualtrics is a web-based survey management platform designed to support survey research, evaluations and other data collection activities.
Participants
We aim to recruit experts from across Australia into the following groups and gain a quota sample of 20 participants in each group: practical/clinical experts (RAC staff including endorsed enrolled nurses, registered nurses, nurse practitioners, nurse managers and occupational therapists working in RAC).
Theoretical/academic experts (geriatricians, geriatric psychiatrists and psychologists, gerontologists, geriatric nurses, occupational therapists and academics specialising in RAC or dementia).
Industry and aged care consumer representatives (eg, Registry of Senior Australians, Leading Aged Services Australia, Australian Association of Gerontology, RAC Provider Organisations, Dementia Australia).
Recruitment
Expert panel members for the Delphi study will be recruited via existing networks of the investigators and by examining current publications to identify experts in the field. Potential participants will respond to the research team if they are interested in taking part to discuss eligibility and informed consent. For the Delphi study, the completion of the survey will be taken as applied consent. Surveys will be completed by participants independently and at their own pace, either online and submitted electronically, or paper based where a return envelope will be provided.
Data analysis
Data from the Delphi survey will be analysed using median (IQR) responses to survey questions. Items with an IQR of ≤1 will be defined as having achieved consensus; an IQR of 0 will be taken to indicate high consensus.36 Between groups, a kappa value between 0.41 and 0.6 will indicate moderate agreement, and between 0.61 and 0.8 will indicate substantial agreement.36 Content validity index (CVI) will be analysed to quantify the level of agreement and increase rigour. CVI can be defined as the degree to which an instrument has an appropriate sample of items for the construct being measured.37 Item content validity (I-CVI) and scale content validity (S-CVI) will be calculated. It has been recommended that a minimum of CVI=0.78 for a scale to be judged as having excellent content validity.38
Refining the MHICare Tool
The performance parameters identified by the Delphi expert panel will be fed back to the CM participants for review and approval in an iterative process to reach a majority approval of the MHICare Tool indicators. At the end of this two-stage process we will have a codesigned mental health indicator tool that is ready for field testing within participating RACFs.
Patient and public involvement
As a research team, we have extensive experience of working with patient and public representatives. This project has been advised on and endorsed by the Community Consumer Involvement Group (CCIG) established by the research team. The CCIG is a research advisory board consisting of people with lived experience of dementia and RAC, who identified initial need for the research project. Nine of them have assisted the study team in helping shape the research proposal especially on the importance and relevance of the research question in ensuring it addresses an important but unmet need in measuring and promoting mental healthcare practices and outcomes for people with dementia living in residential care facilities. The representatives of the group will be asked to comment on the draft indicators developed after the Group Concept Mapping process.
Discussion
The goal of developing the MHICare Tool is to improve current practices and mental health outcomes for people living with dementia in RAC by codesigning a performance measurement tool to aid RACF organisational governance. Stakeholder participation in any BSC development is crucial for implementation success. Engaging RAC industry stakeholders including corporate managers, nurse managers, nursing and care staff, as well as consumer stakeholders including residents and their family members is crucial for relevant and practical indicators reflective of desired outcomes and current processes. The CM method used to develop the indicators has several advantages, including integrating input from multiple sources with different expertise/experience and using rigorous multivariate data analysis techniques to construct maps such as the cluster Go-Zone maps represented in figure 1.33 These maps visually depict the composite thinking of the group and can be used as a guide to develop tools for measurement such as the industry benchmarking MHICare Tool.33 Furthermore, the Delphi process is an established method used to obtain input and consensus35 36 from a group of experts. In this study, experts will be invited from across Australia in the fields of RAC, national aged care industry and consumer groups, as well as research. Delphi process will ensure a set of indicators that are comprehensive and valid.39
In future studies, we plan to test the MHICare Tool within ‘real-world’ RAC settings to determine if the quality indicators are (1) feasible to administer and measure, (2) targeted, specific, acceptable and meaningful and (3) reliably measured.
While indicator tools to improve quality of care in RACF do exist,40 41 there is currently no performance measure with which to monitor and promote mental health for residents with dementia, including those who experience changed behaviours and psychological symptoms of dementia. The MHICare Tool has the potential to measure and improve mental health outcomes for people with dementia within RACF.42 The mental health benchmarking indicator tool has the potential to measure and improve mental health outcomes for people living within RACFs (or long-term care or nursing homes), not only in Australia but may be adapted internationally as appropriate. This may also enable international comparison where appropriate to do so.
Significance
The MHICare Tool will be the first benchmarking tool developed for measuring processes and outputs related to mental healthcare management in Australian RACFs. It will provide an opportunity for monitoring at the industry level as well as the facility level. Future implementation of the Tool has the potential to highlight strengths and gaps of current models of mental healthcare in RAC, instigate improvements at organisational levels to mental health management and create sustainable monitoring behaviours. The MHICare Tool project directly reflects the focus of the Australian Government Royal Commission into Aged Care Quality and Safety, to address complex care needs for residents with an emphasis on mental healthcare, treatment of people with cognitive impairment in aged care and staff training.12 43 44 This project has the potential to instigate a paradigm shift in mental healthcare practices in RAC industry towards optimised policy and care practice planning.
Ethics statements
Patient consent for publication
Consent obtained directly from patient(s).
Contributors RB and ND conceived this project and designed the original protocol. DB, DS and CB further refined the protocol and made appropriate amendments. All five authors were main contributors to the manuscript writing. EB and GB provided research supervision and insight into project development and provided review of protocol and project development.
Funding This study has been funded by a Dementia Australia Research Foundation Project Grant (2022-2024).
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
1 AIHW. People’s Care Needs in Aged Care. Canberra, 2017.
2 Australian Institute of Health and Welfare. Australia's health 2012: the thirteenth biennial health report of the Australian Institute of health and welfare; 2012.
3 Grabowski DC, Aschbrenner KA, Rome VF, et al. Quality of mental health care for nursing home residents: a literature review. Med Care Res Rev 2010; 67: 627–56. doi:10.1177/1077558710362538
4 Sami R. The importance of maintaining connection for the mental health of older adults in residential aged care: lessons from COVID-19. Undergraduate J Politics Policy Soc 2021; 4: 192–216.
5 Burley CV, Casey A-N, Chenoweth L, et al. Views of people living with dementia and their families/care partners: helpful and unhelpful responses to behavioral changes. Int Psychogeriatr 2023; 35: 77–93. doi:10.1017/S1041610222000849
6 Davison TE, Koder D, Helmes E, et al. Brief on the role of psychologists in residential and home care services for older adults. Australian Psychologist 2017; 52: 397–405. doi:10.1111/ap.12209
7 Bhar SS, Silver M, Campbell J, et al. Counselling older adults living in residential aged care settings: four illustrative case studies. Australian Psychologist 2015; 50: 141–7. doi:10.1111/ap.12098
8 Stargatt J, Bhar SS, Davison TE, et al. The availability of psychological services for aged care residents in Australia: a survey of facility staff. Aust Psychol 2017; 52: 406–13. doi:10.1111/ap.12244
9 Gammonley D, Wang X, Simons K, et al. Serious mental illness in nursing homes: roles and perceived competence of social services directors. J Gerontol Soc Work 2021; 64: 721–39. doi:10.1080/01634372.2021.1912241
10 Amare AT, Caughey GE, Whitehead C, et al. The prevalence, trends and determinants of mental health disorders in older Australians living in permanent residential aged care: implications for policy and quality of aged care services. Aust N Z J Psychiatry 2020; 54: 1200–11. doi:10.1177/0004867420945367
11 Australian Institute of health and welfare. Dementia. 2019. Available: https://www.aihw.gov.au/reports/australias-health/dementia
12 Transcript of proceedings; in the matter of the Royal Commision into aged care quality and safety.: hearing before the Royal Commission into aged care quality and safety; 2019.
13 Royal Commission into aged care quality and safety. Final report: care, dignity and respect. Volume 1 summary and recommendations; 2021.
14 Burley CV, Livingston G, Knapp MRJ, et al. Time to invest in prevention and better care of behaviors and psychological symptoms associated with dementia. Int Psychogeriatr 2020; 32: 1–6. doi:10.1017/S104161022000037X
15 Chenoweth L, Stein-Parbury J, Lapkin S, et al. Effects of person-centered care at the organisational-level for people with dementia. A systematic review. PLoS One 2019; 14: e0212686. doi:10.1371/journal.pone.0212686
16 Augl M. Building a conceptual roadmap for systemic change–a novel approach to change management in expert organizations in health care. In: International Conference on Subject-Oriented Business Process Management. New York: Springer, 2012: 43–61. doi:10.1007/978-3-642-29133-3
17 Kaplan RS, Norton DP. The balanced scorecard - measures that drive performance. Harv Bus Rev 1992; 70: 71–9. Available: https://hbr.org/1992/01/the-balanced-scorecard-measures-that-drive-performance-2
18 Lin E, Durbin J. Adapting the balanced scorecard for mental health and Addictions: an inpatient example. Healthc Policy 2008; 3: e160–74.
19 Gauld R, Al-wahaibi S, Chisholm J, et al. Scorecards for health system performance assessment: the New Zealand example. Health Policy 2011; 103: 200–8. doi:10.1016/j.healthpol.2011.05.016
20 Mutale W, Stringer J, Chintu N, et al. Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA intervention: a cluster randomised trial in Zambia. PLoS One 2014; 9: e93977. doi:10.1371/journal.pone.0093977
21 Kollberg B, Elg M. The practice of the balanced scorecard in health care services. Int J Product Perform Management 2011; 60: 427–45. doi:10.1108/17410401111140374
22 Brimelow RE, Amalathas A, Beattie E, et al. The use of balanced Scorecards in mental health services: an integrative review and thematic analysis. J Behav Health Serv Res 2023; 50: 128–46. doi:10.1007/s11414-022-09806-3
23 Rivas C, Tkacz D, Antao L, et al. Automated analysis of free-text comments and dashboard representations in patient experience surveys: a multimethod Co-design study. Health Serv Deliv Res 2019; 7: 1–160. doi:10.3310/hsdr07230
24 de Boer ME, Depla MFIA, Frederiks BJM, et al. Involuntary care–capturing the experience of people with dementia in nursing homes. A concept mapping study. Aging Ment Health 2019; 23: 498–506. doi:10.1080/13607863.2018.1428934
25 Foley J, Ward EC, Burns CL, et al. Speech pathology service enhancement for people with head and neck cancer living in rural areas: using a concept mapping approach to inform service change. Head Neck 2021; 43: 3504–21. doi:10.1002/hed.26850
26 Lord K, Kelleher D, Ogden M, et al. Co-designing complex interventions with people living with dementia and their supporters. Dementia (London) 2022; 21: 426–41. doi:10.1177/14713012211042466
27 QSR International Pty Ltd. Nvivo. 2022. Available: https://lumivero.com/products/nvivo
28 Jackson K, Bazeley P. Qualitative data analysis with Nvivo. Sage; 2019.
29 Allen IE, Seaman CA. Likert scales and data analyses. Quality Progress 2007; 40: 64–5.
30 Chyung SYY, Roberts K, Swanson I, et al. Evidence‐based survey design: the use of a midpoint on the Likert scale. Perf Improv 2017; 56: 15–23. doi:10.1002/pfi.21727
31 GroupWisdom. Group concept mapping. 2022. Available: https://groupwisdom.com/groupconceptmapping
32 Kane M, Trochim WM. Concept mapping for planning and evaluation. In: Concept mapping for planning and evaluation. Thousand Oaks, California: Sage Publications, Inc, 2007. doi:10.4135/9781412983730
33 Trochim W, Kane M. Concept mapping: an introduction to structured conceptualization in health care. Int J Qual Health Care 2005; 17: 187–91. doi:10.1093/intqhc/mzi038
34 Donnelly JP. A systematic review of concept mapping dissertations. Eval Program Plann 2017; 60: 186–93. doi:10.1016/j.evalprogplan.2016.08.010
35 Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends Methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014; 67: 401–9. doi:10.1016/j.jclinepi.2013.12.002
36 Jorm AF. Using the Delphi expert consensus method in mental health research. Aust N Z J Psychiatry 2015; 49: 887–97. doi:10.1177/0004867415600891
37 Polit DF, Beck CT. Nursing Research: Principles and Methods. Philadelphia: Lippincott Williams & Wilkins, 2004.
38 Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007; 30: 459–67. doi:10.1002/nur.20199
39 Hibbert PD, Molloy CJ, Wiles LK, et al. Designing clinical indicators for common residential aged care conditions and processes of care: the Caretrack aged development and validation study. Int J Qual Health Care 2022; 34: mzac033. doi:10.1093/intqhc/mzac033
40 Carpenter IH. Using interRAI assessment systems to measure and maintain quality of long term care. A good life in old age? monitoring and improving quality in long-term care. OECD health policy studies. OECD Publishing; 2013.
41 Courtney M, O’Reilly MT, Edwards H, et al. Development of a systematic approach to assessing quality within Australian residential aged care facilities: the clinical care indicators tool. Aust Health Rev 2007; 31: 582–91. doi:10.1071/ah070582
42 McDonald B. A review of the use of the balanced scorecared in healthcare. Newcastle: BMCD Consulting, 2012.
43 Dyer S, van den Berg M, Brown A, et al. Review of innovative models of aged care: Royal Commission into aged care quality and safety. 2020.
44 Purser K, Lewis B, Kruck L, et al. Submission to the Royal Commission into aged care quality and safety; 2019. Available: https://australian.physio/sites/default/files/Submissions/APA_Aged_Care_Royal_Commission_Submission.pdf
© 2024 Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. http://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ . Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Brooks, Deborah 1 ; Sriram, Deepa 1 ; Burley, Claire 2 ; Beattie, Elizabeth 3 ; Byrne, Gerard 4 ; Dissanayaka, Nadeeka 5 1 University of Queensland Centre for Clinical Research (UQCCR), Herston, Queensland, Australia
2 UNSW Medicine & Health Lifestyle Clinic, School of Health Sciences, University of New South Wales, Sydney, New South Wales, Australia
3 Faculty of Health, School of Nursing, QUT, Brisbane, Queensland, Australia
4 Academy of Psychiatry, UQ Faculty of Medicine, Herston, Queensland, Australia
5 Centre for Clinical Research, University of Queensland Faculty of Medicine, Herston, Queensland, Australia