Building a practice-based research network for healthcare integration: a protocol paper for a mixed-method project

Abstract

Introduction

Practice-based research networks (PBRNs) are sustained collaborations between healthcare professionals, researchers and members of the community that develop, conduct and report on research relevant to local needs. While PBRNs have traditionally been focused towards primary care practices and their patients, there has been increasing interest in how they may help facilitate healthcare integration. Yet, little is known on the ways in which PBRNs can best integrate with the broader healthcare system, in particular Advanced Health Research and Translation Centres. The overall project aim is to build a sustainable collaboration between a PBRN and an Advanced Health Research and Translation Centre to generate a research platform suitable for planning, undertaking and translating research to improve care across the healthcare continuum.

Methods and analysis

We will use a developmental evaluation design. Our iterative approach will be informed by a programme logic model and consists of: preparation work (pre-implementation assessment, literature review, community and stakeholder engagement), adaptation and building for a sustainable collaboration (strategy for recruitment and sustainment of members) and planning for network action (designing and implementing priority initiatives, monitoring and follow-up).

Ethics and dissemination

This project was approved by the Monash Health ethics committee (ERM Reference Number: 76281; Monash Health Ref: RES-21-0000-392L) and the Monash University Human Research ethics committee (Reference Number: 29786). Dissemination will take place via various channels, including relevant national and international committees and conferences, peer-reviewed journals and social media. Continuous dissemination to and communication with all participants in this project as well as other relevant stakeholders will help strengthen and sustain the network.

Correspondence to Professor Grant Russell; [email protected]

Strengths and limitations of this study

• A strength of this study is that we are engaging with end-users and policymakers through the project, from conceptualisation to uptake.

• The research team will ensure that clinicians are involved in the whole process by building on their many years of experience in engaging general practitioners in the region.

• A limitation of this project is that this is a case study in one context.

Background

Practice and research often operate within separate silos. Practice-based research networks (PBRNs) have been seen as one approach to break down the barriers between these two worlds.^1–3 PBRNs are sustained collaborations between primary care professionals, researchers, members of the community and policymakers that develop, conduct and report on research relevant to local healthcare needs.^{4 5} They have shown to act as vehicles for conducting primary care research in a range of Western nations.^{1 6}

PBRNs have the potential to act as research laboratories for generating research-based solutions to questions that matter to local primary care professionals and to facilitate healthcare improvement.^{7 8} They have been seen as critical to the process of speeding up the translation of research into practice and play a central role in optimising the quality of care in the local setting.^{4 9} They can reinforce the formation of new partnerships linking the needs of local communities, healthcare professionals, academics, funding agencies and policymakers.⁹

While PBRNs have traditionally been focused around the needs of primary care providers and their patients, there has been increasing interest in their potential to contribute to broader healthcare integration.¹ In the context of primary care, integrated care represents a network of multiple professionals and organisations across the health and social care system providing accessible, comprehensive and coordinated services to a population in a community.¹⁰ Such integration has been encouraged as a means to improve access, quality and continuity of services in a more efficient way, especially for individuals with complex needs.^10–14

The concept of integrated care has become a focus of Australia’s National Health and Medical Research Council’s Advanced Health Research and Translation Centre (AHRTC) initiative.¹⁵ AHRTCs, very similar to Academic Health Science Centres in other nations are designed to improve the health and well-being of patients and communities through strengthening collaboration between health services and research institutions, building research and research translation capacity and promoting health service leadership focused on priorities relevant to both health services and surrounding populations.

There has been increasing interest in the potential for PBRNs to represent a point of reference for research and collaboration between teaching hospitals, universities, community-based services and primary care practices. Monash Partners, located in Eastern and South East Melbourne in Australia aims to facilitate such connections between researchers, clinicians and the community to innovate for better health and well-being.

While many PBRNs have been established in Australia¹⁶ and overseas, little is known about the ways in which PBRNs can best integrate with the broader healthcare system, in particular AHRTC.

This project arose from a collaboration between Monash Partners and the Department of General Practice at Monash University, and its associated PBRN, the Monash practice-based Research Network (MonReN). The overall project aim is to build a sustainable collaboration between a PBRN (MonReN) and an AHRTC (Monash Partners) to generate a research platform suitable for planning, undertaking and translating research to improve care across the healthcare continuum.

Objectives

Over the course of 18 months we will:

• Map the current environment within the Monash Partners region.

• Identify and engage key stakeholders.

• Learn from international best practice.

• Tailor the current governance strategy for a contemporary, translational environment.

• Design research projects that capitalise on the benefits of a PBRN-AHRTC collaboration.

Methods and analysis

Design

We will use a developmental evaluation design^{17 18} while building the foundations for a sustainable PBRN-AHRTC collaboration.

The developmental evaluation design is an iterative approach in which researchers gather data about the factors affecting a programme’s functioning within a complex environment. When an intervention takes place under complex conditions, numerous factors interact and influence each other, making it impossible to predict what will happen as the intervention moves forward.^{17 18} The approach recognises the importance of adapting programmes to the circumstances of complex social environments.^19–21 It is especially useful when adapting a programme to emerging conditions, modifying approaches for use in new contexts, developing scalable innovations and generating feedback about an innovation as it moves forward.^{19 22}

Setting

The project takes place in the catchment area of Monash Partners, one of Australia’s 10 accredited AHRTC, which is in the South East and Eastern regions of Melbourne in the state of Victoria. Monash Partners represent four state funded health services (Alfred Health, Monash Health, Peninsula Health and Eastern Health), two private hospitals (Cabrini Health and Epworth HealthCare) three medical research institutes (the Burnet Institute, Hudson Institute, Baker Heart and Diabetes Institute), Monash University and two associate partners (La Trobe University and Latrobe Regional Hospital).

The Monash Partners catchment area includes about 3.2 million Australians. It is a diverse and rapidly growing population. The ethnic and cultural diversity includes 49% of community members born overseas and 57% with both parents born overseas. About 3% of the population being Indigenous. Over 100 languages are spoken, 58% of community members speak only English at home and there are more children, less professionals and lower income in the Monash Partners catchment area in comparison to national averages. The catchment area also hosts the highest density of refugees nationally.

An iterative approach with multiple phases

Our iterative approach consists of three phases (see box 1) and is informed by a programme logic model (see figure 1). Our preliminary logic model is based on previous PBRN literature^{1 23} and represents the mechanisms and potential consequences of the activities within each phase of our project.^24–26 Throughout the project, our logic model will be continuously refined. The project started at the end of January 2021 and is expected to end by July 2022.

Box 1

Overview of the three phases of the project

Phase 1: Preparation work

 1.1 Pre-implementation assessment.

 1.2 Literature review.

 1.3 Community and stakeholder engagement.

  1.3.1 Identifying and engaging key stakeholders.

  1.3.2 Defining governance structure.

  1.3.3 Assessing stakeholders’ needs.

  1.3.4 Collaborative priority setting.

Phase 2: Adaptation and building for a sustainable practice-based research network (PBRN)-Advanced Health Research and Translation Centre (AHRTC) collaboration

 2.1 Strategy for recruitment and sustainment of members.

Phase 3: Planning for PBRN-AHRTC action

 3.1 Designing and implementing priority initiatives.

 3.2 Monitoring and follow-up.

Figure 1. The preliminary logic model for the project. PBRN, practice-based research network; GP, general practitioner.

Phase 1: preparation work

1.1 Pre-implementation assessment

We will assess key contextual and organisational features likely to influence sustainability and impact of the PBRN-AHRTC collaboration. In consultation with key stakeholders, we will map the current environment, as recommended by previous research,^27–29 including community practice capacity and potential, Monash Partners’ capacity and data availability.

MonReN has been engaging with community general practices for almost a decade. We will map past and current involvement of practices in teaching and research and willingness to engage in a PBRN in the future (including what the participants would want from a PBRN).

Monash Partners has access to key opportunities for research capacity building for researchers and clinicians in the region. We will scope the potential for community primary care clinicians and researchers to benefit from these opportunities.

We will avoid duplication of effort by building on the capacity of Monash Partners, the Primary Health Networks and existing links between the Department of General Practice and Monash Partners. We will map key data sources available in our region and nationally, such as data held by data providers, the Australian Government Pharmaceutical Benefits Scheme, hospital admission databases, emergency department utilisation and Medical Benefit Schedule. This will also incorporate mapping and documentation of potential data platforms including assessment of capability and primary care interest.

1.2 Literature review

Phase 1 will be complemented by a scoping review of the literature to identify best practices regarding PBRNs. The review will seek to understand how to develop a PBRN, how to sustain it and how to evaluate PBRN performance. Besides this broad approach, we are also more specifically interested in how primary care PBRNs have ensured integration with the broader healthcare system.

This scoping review will be conducted in accordance with the framework presented by Arksey and O’Malley, as updated by Levac et al.^30–32 and will comply with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Extension for Scoping Reviews checklist.³³

Search strategy

The following databases will be searched for relevant literature: Ovid MEDLINE, Embase, CINAHL and Scopus. We will include studies concerning primary care Practice-Based Research Networks and collaborations with the broader healthcare system. We define ‘primary care’ as settings related to healthcare service delivery to individuals within the community.³⁴ The search terms will include ‘Practice-based Research Network*’, ‘PBRN*’, ‘practice-based research’, ‘practice research network*’ and ‘integrated care’ (See online supplemental appendix 1).

Study selection

This scoping review will consider for inclusion papers published after 2000 that have quantitative, qualitative and mixed-methods study designs, as well as commentaries and editorials. Studies solely situated in the hospital setting/secondary care will be excluded. Reviews will not be eligible but we will screen reference lists for eligible studies that were not identified by our search strategy. Grey literature will purposely be excluded to optimise the veracity of the findings.

We will use an iterative approach to select studies for inclusion. All identified records will be collated and uploaded into Covidence (Veritas Health Innovation, Melbourne, Australia). Following a pilot test, titles and abstracts will be screened independently by two reviewers for assessment against the inclusion criteria. The researchers will meet several times during the selection process to create a shared understanding of the inclusion criteria and to discuss any challenges. The full text of selected citations will be assessed in detail by two independent reviewers. Any disagreements that will arise between the reviewers at each stage of the selection process will be resolved through discussion or with a third reviewer.

Charting the data

Once all included studies are identified, a data extraction tool will be created and pilot tested. The data extraction will be done by multiple researchers and frequent discussions will help to get a shared understanding of the data. The findings of this review will help inform the next steps in our project.

1.3 Community and stakeholder engagement

1.3.1 Identifying and engaging key stakeholders

We will develop, in consultation with our partners a skills matrix of all relevant areas of expertise and stakeholder roles, including General Practitioners (GPs), GP practice managers/owners, GP practice nurses, Monash Partners representatives, hospital clinician researchers, health service-based research representatives, Primary Health Networks representatives, public health researchers, professional GP organisation representative, policymakers (community health services or state government), representatives of local, state or federal government and patient representatives.

This matrix will lead to an initial list of potential stakeholders, who will be recruited using the snowballing recruitment method,³⁵ starting with the members of the investigators team approaching key contacts who may be interested in collaborating. Engaged stakeholders will be asked to suggest other relevant candidates. Initial contact will take place via phone or email to assess interest and consent will be requested for the project manager to send a follow-up email including an invitation, a participant information letter and Informed Consent Form.

Patient and public involvement

Community members will be involved at all levels of the project, including the governance. We have allocated funds for engaging community members in line with the principles of equity, trust and transparency. Monash Partners has extensive experiences in public and patient involvement. We can build on their expertise and have a patient representative collaborating on this project. Throughout this project we strive for a participatory level of community engagement.³⁶ This includes strong bidirectional relationships, equally shared decision-making and strong emphasis on partnership building.

1.3.2 Defining governance structure

The eight investigators (including academic GPs, public health and primary care researchers and implementation scientists) will monitor the project’s progress and make strategic decisions together with a stakeholder advisory committee (comprising general practitioners, community members, policymakers, a public health researcher familiar with primary care, a leader from another PBRN and a representative from Monash Partners and MonReN). The stakeholder advisory committee will also help facilitate the translation of project findings into policy and practice.

1.3.3 Assessing stakeholders’ needs

We want to gain an insight into key stakeholders’ needs and relevant contextual factors in order to make the integrated PBRN as suitable as possible to the local context and enhance a bottom-up developmental process.

Data collection

Semi-structured interviews will be conducted with key stakeholders as identified in 1.3.1 and this qualitative research will be informed by the COREQ (Consolidated criteria for Reporting Qualitative research) checklist.³⁷ The interview guide was developed by the research team and a pilot interview will help to refine the interview questions. The guide contains the following main topics: participants’ personal experiences with practice-based research and data-led healthcare improvements, participants’ perspectives on potential partnerships between healthcare professionals, academics and data providers and stakeholders’ needs about a research network on integrated care. Several versions of the interview guide were created to adjust the questions based on the background and expertise of the interviewees: Participants working in GP practices, other healthcare staff, community members and academics. All interview guides are included in online supplemental appendix 2. Interviews will be audio-recorded, transcribed and de-identified through the use of ID numbers or pseudonyms for people and places.

Data analysis

Data analysis will be iterative and based on the principles of thematic analysis.³⁸ After an inductive coding process, recurring themes will be identified. NVivo software V.20 will be used to facilitate the coding process. Based on sample size recommendations for qualitative research,³⁹ an initial sample of 15 participants will be recruited. This initial data set will be assessed for thematic saturation. If no new themes emerge, data saturation is deemed to be reached and no additional individuals will be recruited. If saturation is not achieved, a further two interviews will be conducted until data saturation is reached.

The key findings of the interviews will be summarised and reviewed by the investigator team in order to provide a deeper understanding, identify priority gaps and offer perspectives on how the contextual factors might influence the design of our integrated PBRN. The key findings will also be used to discuss during the collaborative decision-making process around priorities.

1.3.4 Collaborative priority-setting

We want to involve as many local stakeholders as possible in the decision-making process around priorities for the integrated PBRN. These priorities could be research related (eg, stakeholders’ interest in research in the area of infectious diseases) or non-research related (eg, professional development opportunities within the network).

Pre-reading

One week before the actual group discussion, participants will receive a summary of key findings from our scoping review and interviews. This information will be discussed at the start of the group meeting.

Group discussion

The decision-making process will take place during a group discussion, based on the principles of the nominal group technique.⁴⁰ The process will involve four stages: silent idea generation, small group discussions, big group discussion and prioritisation.⁴¹

Each participant will independently record their responses to the following questions before sharing them with a small group:

• What kind of collaboration do you foresee between general practices, other healthcare professionals, researchers and data networks?

• Which research priorities do you suggest for such a collaboration?

• Which non-research-related aspects are important in order to make this collaboration as successful as possible?

All ideas will be recorded by a facilitator who will then lead a group discussion where each idea is discussed, grouped and clarified. Afterwards, individuals can vote privately to indicate their priorities and results are anonymously fed back to the group.^40–42 A detailed guide has been developed to structure the group discussion (see online supplemental appendix 3).

It is anticipated that 30–40 individuals participate in the group discussion and it will be approximately 2 hours in duration. It is our intention to have a face-to-face meeting at an accessible location in the South East region of Melbourne, if allowed by the governmental public health regulations at that time. We will also have an online alternative, if a face-to-face meeting is not allowed. In that case, we will use a virtual platform that has ‘break out rooms’ to have one-on-one or small group discussions and has the possibility for individual voting.

Evaluation

Consensus will be reached if 80% of all participants agree on the prioritisation of both research and non-research-related aspects. Pertinent aspects of the discussion will be recorded by note takers and all items on which agreement was reached will be summarised in a consensus document. A short evaluation form will be distributed at the end of the group discussion (see online supplemental appendix 4). The data will be used to give some context to the findings of the group discussion and be used to improve future group discussions.

The prioritisation of both research topics and non-research-related aspects for the PBRN-AHRTC collaboration will be used to inform the next stage of the project.

Phase 2: adaptation and building for a sustainable PBRN-AHRTC collaboration

2.1 Strategy for recruitment and sustainment of members

Using the outputs from Phase 1, we will tailor the PBRN-AHRTC collaboration to meet the needs of the region. We will engage with general practices, community members and members of the AHRTC to build a structured approach to membership. Additionally, marketing strategies and communication channels will be developed to maintain communication with and among all members.

The strengths of clinicians and academics will be showcased in up to four networking and education events. The events will be open to those interested in joining. It will highlight care and research across the healthcare continuum. It is also anticipated that Monash Partners will have research capacity building opportunities that clinicians and researchers will be able to participate in and could incorporate formal activities for which GPs can claim Continuing Professional Development points.

Phase 3: planning for PBRN-AHRTC action

3.1 Designing and implementing priority initiatives

The last part of the 18 months will be used to design and implement priority initiatives.

The two highest research priorities, as identified in the group discussion, will be co-designed with community members and implemented in the local region. This will allow the integrated PBRN to pragmatically implement the process for moving from an identified research priority to actual research in practice. Within the first 12 months we will plan the projects with a view to carrying them out in the following year. Funding opportunities will be sought as certain priority initiatives might require extra funding for implementation and scale up.

Besides the non-research-related priorities that will be identified during the group discussion, the integrated PBRN also aims to target two non-research-related priorities, such as organising networking and educational events.

3.2 Monitoring and follow-up

The set-up process will be evaluated after the first year. The findings from the literature review will help us develop suitable key indicators to assess, monitor and follow-up on performance of the PBRN-AHRTC collaboration. Performance indicators might relate to the number of practices engaged, attendance at the events, ability to communicate with members, development of the practice database system and ability to develop priority initiatives.

Discussion

The outcomes of all the iterative phases in this developmental approach will be used to inform our strategy for building a sustainable collaboration between a primary care PBRN and an AHRTC in our local context. The developmental evaluation design will help us to adapt our strategies and activities to the circumstances of the complex social environment in our region.

This project aims to achieve a research platform for designing, undertaking and reporting on research to improve care within the whole healthcare system. Expected outputs, early outcomes and late outcomes are outlined in our preliminary logic model (see figure 1).

Potential impact

For community general practice, this collaboration provides a mechanism for undertaking research that answers questions of relevance to them and their patients plus a better connection to the broader healthcare system. For Monash Partners, this collaboration will offer key insights into the critical connection between primary and secondary care, improve mutual understanding and showcase national leadership in this emerging area of PBRN practice. Ultimately, it is hoped that this collaboration will improve population health, patient experiences, cost-efficiency, the work of healthcare providers and better integrated care.

The value of this platform is the long-term relationships between the Department of General Practice, community general practice, Monash Partners and the broader Monash Partners’ network. This is a two-way relationship that aims to be responsive to the needs of our region. Little has been published on the ways in which primary care PBRNs can best integrate with the broader healthcare system, in particular AHRTC. This project will showcase the collaboration between general practices and an AHRTC as an exemplar PBRN-AHRTC collaboration. It will be of interest to other PBRNs and AHRTCs as well as broader health organisations around Australia and internationally.

Ethics and dissemination

This project was approved by the Monash Health ethics committee (ERM Reference Number: 76281; Monash Health Ref: RES-21-0000-392L) and the Monash University Human Research Ethics Committee (Reference Number: 29786).

At the end of the project, all participants will receive a two-page summary of the research that they were involved in. Any feedback from participants will be taken into consideration prior to publication of the data. We will also disseminate the findings via presentations at relevant local, national and international committees and conferences, peer-reviewed journals, through social media and various communication channels of Monash Partners and the Department of General Practice, Monash University. This partnership approach will also enable broader dissemination through the networks of these partners (across university, health and policy).

Dissemination within the newly established PBRN-AHRTC collaboration will be crucial for sustaining the relationship. Given that PBRNs ideally foster longitudinal relationships and promote ongoing collaborations between researchers, healthcare providers and other community members, continued efforts will be required to sustain this collaboration.⁴³ Continuous dissemination and communication of research projects as well as other non-research-related activities will help strengthen and sustain the connection between all stakeholders.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Twitter @SamanthaC_22, @LizSturgiss, @grantrussell17

Contributors GR, SPC, CB, EAS, DM, MDL-S and TS-S designed the study. SP provided further design input following funding. All authors were involved in the planning of the study. SP and GR led the conduct of the work and the writing of this manuscript. All other authors provided comments on drafts and approved the final version.

Funding This work was supported by the Medical Research Future Fund (MRFF) via the Monash Partners Academic Health Research Translation Centre in December 2019. Grant/award number: N/A.

Competing interests None declared.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Dania A, Nagykaldi Z, Haaranen A, et al. A review of 50-Years of international literature on the internal environment of building practice-based research networks (PBRNs). J Am Board Fam Med 2021; 34: 762–97. doi:10.3122/jabfm.2021.04.200595 http://www.ncbi.nlm.nih.gov/pubmed/34312269

² Rhyne RL, Fagnan LJ. Practice-based research network (PBRN) engagement: 20+ years and counting. J Am Board Fam Med 2018; 31: 833–9. doi:10.3122/jabfm.2018.06.180274 http://www.ncbi.nlm.nih.gov/pubmed/30413538

³ Hickner J, Green LA. Practice-based research networks (PBRNs) in the United States: growing and still going after all these years. J Am Board Fam Med 2015; 28: 541–5. doi:10.3122/jabfm.2015.05.150227 http://www.ncbi.nlm.nih.gov/pubmed/26355124

⁴ Westfall JM, Roper R, Gaglioti A, et al. Practice-based research networks: strategic opportunities to advance implementation research for health equity. Ethn Dis 2019; 29: 113–8. doi:10.18865/ed.29.S1.113 http://www.ncbi.nlm.nih.gov/pubmed/30906158

⁵ Pearce KA, Love MM, Barron MA, et al. How and why to study the practice content of a practice-based research network. Ann Fam Med 2004; 2: 425–8. doi:10.1370/afm.133 http://www.ncbi.nlm.nih.gov/pubmed/15506575

⁶ Dolor RJ, Campbell-Voytal K, Daly J, et al. Practice-based research network research good practices (PRGPs): summary of recommendations. Clin Transl Sci 2015; 8: 638–46. doi:10.1111/cts.12317 http://www.ncbi.nlm.nih.gov/pubmed/26296309

⁷ Green LA, White LL, Barry HC, et al. Infrastructure requirements for practice-based research networks. Ann Fam Med 2005; 3 Suppl 1: S5–11. doi:10.1370/afm.299 http://www.ncbi.nlm.nih.gov/pubmed/15928219

⁸ Hayes H, Burge S. Creating a practice-based research network from scratch: where do I begin? Prog Community Health Partnersh 2012; 6: 369–80. doi:10.1353/cpr.2012.0033 http://www.ncbi.nlm.nih.gov/pubmed/22982850

⁹ Gaglioti AH, Werner JJ, Rust G, et al. Practice-based research networks (PBRNs) bridging the gaps between communities, Funders, and policymakers. J Am Board Fam Med 2016; 29: 630–5. doi:10.3122/jabfm.2016.05.160080 http://www.ncbi.nlm.nih.gov/pubmed/27613796

¹⁰ Valentijn PP, Boesveld IC, van der Klauw DM, et al. Towards a taxonomy for integrated care: a mixed-methods study. Int J Integr Care 2015; 15: e003. doi:10.5334/ijic.1513 http://www.ncbi.nlm.nih.gov/pubmed/25759607

¹¹ Kodner DL. All together now: a conceptual exploration of integrated care. Healthc Q 2009; 13 Spec No: 6–15. doi:10.12927/hcq.2009.21091 http://www.ncbi.nlm.nih.gov/pubmed/20057243

¹² Suter E, Oelke N, Adair C, et al. Ten key principles for successful health systems integration. Hcq 2009; 13: 16–23. doi:10.12927/hcq.2009.21092

¹³ Cifuentes M, Fernald DH, Green LA, et al. Prescription for health: changing primary care practice to foster healthy behaviors. Ann Fam Med 2005; 3 Suppl 2: S4–11. doi:10.1370/afm.378 http://www.ncbi.nlm.nih.gov/pubmed/16049083

¹⁴ Angus L, Valentijn PP. From micro to macro: assessing implementation of integrated care in Australia. Aust J Prim Health 2018; 24: 59–65. doi:10.1071/PY17024

¹⁵ Recognised Health Research and Translation Centres | NHMRC. Available: https://www.nhmrc.gov.au/research-policy/research-translation/recognised-health-research-and-translation-centres

¹⁶ Temple-Smith M, Novy K, Manski-Nankervis JA. A Snapshot of Australian Practice Based Research Networks in Primary Care. In: A report from the Melbourne academic centre. Australia, 2021.

¹⁷ Conklin J, Farrell B, Ward N, et al. Developmental evaluation as a strategy to enhance the uptake and use of deprescribing guidelines: protocol for a multiple case study. Implementation Sci 2015; 10: 91. doi:10.1186/s13012-015-0279-0

¹⁸ Rey L, Tremblay M, Brousselle A. Managing tensions between evaluation and research: illustrative cases of developmental evaluation in the context of research. American Journal of Evaluation 2014; 35: 45–60. doi:10.1177/1098214013503698

¹⁹ Patton MQ. Developmental evaluation: applying complexity concepts to enhance innovation and use. New York: Guilford, 2011.

²⁰ Hummelbrunner R. Systems thinking and evaluation. Evaluation 2011; 17: 395–403. doi:10.1177/1356389011421935

²¹ Kania A, Patel AB, Roy A, et al. Capturing the complexity of evaluations of health promotion interventions: a scoping review. Canadian Journal of Program Evaluation 2013; 27: 65–91.

²² Wimbush E, Watson J. An evaluation framework for health promotion: theory, quality and effectiveness. Evaluation 2000; 6: 301–21. doi:10.1177/135638900000600302

²³ Hayes H, Parchman ML, Howard R. A logic model framework for evaluation and planning in a primary care practice-based research network (PBRN). J Am Board Fam Med 2011; 24: 576–82. doi:10.3122/jabfm.2011.05.110043 http://www.ncbi.nlm.nih.gov/pubmed/21900441

²⁴ Stinson S, Wilkinson C. Creating a successful clinical extern program using a program planning logic model. J Nurses Staff Dev 2004; 20: 140–4. doi:10.1097/00124645-200405000-00007 http://www.ncbi.nlm.nih.gov/pubmed/15201835

²⁵ Kaplan SA, Garrett KE. The use of logic models by community-based initiatives. Eval Program Plann 2005; 28: 167–72. doi:10.1016/j.evalprogplan.2004.09.002

²⁶ Knowlton LW, Phillips CC. The logic model guidebook: better strategies for great results. Los Angeles: Sage, 2009.

²⁷ DeVoe JE, Likumahuwa S, Eiff MP, et al. Lessons learned and challenges ahead: report from the OCHIN safety net West practice-based research network (PBRN). J Am Board Fam Med 2012; 25: 560–4. doi:10.3122/jabfm.2012.05.120141 http://www.ncbi.nlm.nih.gov/pubmed/22956690

²⁸ Cole AM, Stephens KA, Keppel GA, et al. Extracting electronic health record data in a practice-based research network: processes to support translational research across diverse practice organizations. EGEMS 2016; 4: 1206. doi:10.13063/2327-9214.1206 http://www.ncbi.nlm.nih.gov/pubmed/27141519

²⁹ Calmbach WL, Ryan JG, Baldwin L-M, et al. Practice-based research networks (PBRNs): meeting the challenges of the future. J Am Board Fam Med 2012; 25: 572–6. doi:10.3122/jabfm.2012.05.120064 http://www.ncbi.nlm.nih.gov/pubmed/22956692

³⁰ Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8: 19–32. doi:10.1080/1364557032000119616

³¹ Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010; 5: 69. doi:10.1186/1748-5908-5-69 http://www.ncbi.nlm.nih.gov/pubmed/20854677

³² O'Brien KK, Colquhoun H, Levac D, et al. Advancing scoping study methodology: a web-based survey and consultation of perceptions on terminology, definition and methodological steps. BMC Health Serv Res 2016; 16: 305. doi:10.1186/s12913-016-1579-z http://www.ncbi.nlm.nih.gov/pubmed/27461419

³³ Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169: 467–73. doi:10.7326/M18-0850 http://www.ncbi.nlm.nih.gov/pubmed/30178033

³⁴ Muldoon LK, Hogg WE, Levitt M. Primary care (PC) and primary health care (PHC). Can J Public Health 2006; 97: 409–11. doi:10.1007/BF03405354

³⁵ Savin-Baden M, Major C. Qualitative research: the essential guide to theory and practice. London: Routledge, 2013.

³⁶ Yuen T, Park AN, Seifer SD, et al. A systematic review of community engagement in the US environmental protection agency's extramural research Solicitations: implications for research Funders. Am J Public Health 2015; 105: e44–52. doi:10.2105/AJPH.2015.302811 http://www.ncbi.nlm.nih.gov/pubmed/26469656

³⁷ Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007; 19: 349–57. doi:10.1093/intqhc/mzm042 http://www.ncbi.nlm.nih.gov/pubmed/17872937

³⁸ Braun V, Clarke V. What can "thematic analysis" offer health and wellbeing researchers? Int J Qual Stud Health Well-being 2014; 9: 26152. doi:10.3402/qhw.v9.26152 http://www.ncbi.nlm.nih.gov/pubmed/25326092

³⁹ Francis JJ, Johnston M, Robertson C, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health 2010; 25: 1229–45. doi:10.1080/08870440903194015 http://www.ncbi.nlm.nih.gov/pubmed/20204937

⁴⁰ Humphrey-Murto S, Varpio L, Gonsalves C, et al. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach 2017; 39: 14–19. doi:10.1080/0142159X.2017.1245856 http://www.ncbi.nlm.nih.gov/pubmed/27841062

⁴¹ McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm 2016; 38: 655–62. doi:10.1007/s11096-016-0257-x http://www.ncbi.nlm.nih.gov/pubmed/26846316

⁴² Waggoner J, Carline JD, Durning SJ. Is there a consensus on consensus methodology? descriptions and recommendations for future consensus research. Acad Med 2016; 91: 663–8. doi:10.1097/ACM.0000000000001092 http://www.ncbi.nlm.nih.gov/pubmed/26796090

⁴³ Goldstein KM, Vogt D, Hamilton A, et al. Practice-based research networks add value to evidence-based quality improvement. Healthc 2018; 6: 128–34. doi:10.1016/j.hjdsi.2017.06.008 http://www.ncbi.nlm.nih.gov/pubmed/28711505