Mapping continuous learning using social network research: a social network study of Australian Genomics as a Learning Health System

Abstract

Objectives

To explore a macrolevel Learning Health System (LHS) and examine if an intentionally designed network can foster a collaborative learning community over time. The secondary aim was to demonstrate the application of social network research to the field of LHS.

Design

Two longitudinal online questionnaires of the Australian Genomics learning community considering relationships between network members at three time points: 2016, 2018, 2019. The questionnaire included closed Likert response questions on collaborative learning patterns and open-response questions to capture general perceptions of the community. Social network data were analysed and visually constructed using Gephi V.0.9.2 software, Likert questions were analysed using SPSS, and open responses were analysed thematically using NVivo.

Setting

Australian Genomic Health Alliance.

Participants

Clinicians, scientists, researchers and community representatives.

Results

Australian Genomics members highlighted the collaborative benefits of the network as a learning community to foster continuous learning in the ever-evolving field of clinical genomics. The learning community grew from 186 members (2016), to 384 (2018), to 439 (2019). Network density increased (2016=0.023, 2018=0.043), then decreased (2019=0.036). Key players remained consistent with potential for new members to achieve focal positions in the network. Informal learning was identified as the most influential learning method for genomic practice.

Conclusions

This study shows that intentionally building a network provides a platform for continuous learning—a fundamental component for establishing an LHS. The Australian Genomics learning community shows evidence of maturity and sustainability in supporting the continuous learning culture of clinical genomics. The network provides a practical means to spread new knowledge and best practice across the entire field. We show that intentionally designed networks provide the opportunity and means for interdisciplinary learning between diverse agents over time and demonstrate the application of social network research to the LHS field.

Correspondence to Dr Louise A Ellis; [email protected]

Strengths and limitations of this study

• Successful demonstration of the application of social network research as a key tool to visualise and quantitatively analyse social-based learning in the Learning Health System (LHS) field.

• Social network methods were supplemented with closed-response and open-response survey questions to identify how learning occurs in the network.

• Focused on only one component of an LHS (continuous learning), and the survey approach provides only a snapshot a network that is inherently dynamic.

• Data collection for the year 2016 was retrospectively captured in 2018, potentially resulting in recall bias.

Background

The vision of a Learning Health System (LHS) is a system that can harness the power of data to learn from multiple sources including patients, and rapidly generate and feed new knowledge back to diverse agents for better decision-making and continuous cycles of improvement.¹ It is a system where big data and artificial intelligence are used to synthesise clinical information from electronic health records and create new knowledge to improve patient care. More specifically, an LHS has been defined as an integrated system which brings together ‘Science and Informatics’, ‘Patient-Clinician Partnerships’, ‘Incentives’, and a ‘Continuous Learning Culture’.¹ The concept of an LHS was first introduced in the USA through a series of publications by the Institute of Medicine (now the National Academy of Medicine).^1–3 Since their first report was published in 2007,² the concept has gained traction internationally^4–6 and is now recognised as an imperative to increase high-quality care and limit low-value care, harm and waste in the health system.^{7 8}

The idealised LHS is an ultra-large-scale, ultracomplex system of systems, combining all data and learning from each interaction in any and all parts of the health system.³ In practice, LHSs manifest at different levels (figure 1)⁹ (micro, meso, and macro) and serve different functions. At the microlevel, an LHS can support personalised care; at the mesolevel, it can help the organisation optimise their care delivery; at the macrolevel, an LHS can inform and be informed by policy and planning; while in the meantime amalgamating all data in the health system at the ultra-large scale. The idealised ultra-large-scale LHS remains aspirational,⁸ although we are beginning to see the realisation of LHSs at more nuanced levels of the health system.^4–6 A recent review by Ellis et al¹⁰ found 76 empirical applications of LHS examples across 11 countries, with the vast majority of research being conducted at the microlevels and mesolevels, and an identified need for further research at the macrolevel. This review also identified the need for innovative methods, both quantitative and qualitative, to move the LHS field forward.

Figure 1. The levels within a Learning Health System.

Of the emerging or established LHS examples, most favour data capabilities of the LHS over behavioural factors to support change and learning activities,¹¹ such as continuous and collaborative learning.⁶ Establishing a continuous learning culture is an important component of LHSs,⁴ and one of the four pillars of an LHS as defined by the Institute of Medicine.² Common ways to establish this culture are through practice-based research networks and learning communities.⁶ These may be networks or collaborative structures that bring together different patients, healthcare professionals, and managers to share knowledge and contribute to the LHS goal of continuous quality improvement and learning.¹² They are dynamic, non-hierarchical and ‘actor-oriented’.¹² For example, the Advanced Cardiac Therapies Improving Outcomes Network (ACTION) is a learning network developed to improve the outcomes of paediatric patients with congenital heart disease with end-stage heart failure.¹³ It is a multicentre learning health network that harnesses the benefits of an LHS for this particular population. The network’s activities include: building a multi-institutional data repository, developing protocols and educational materials, and conducting quality improvement across the network/LHS. A key aim of ACTION is to establish grounds for collaboration between all stakeholders: providers, patients, families and industry personnel. While there are several examples of networks to support collaborative and continuous learning in LHSs in the literature,¹² there is a lack of knowledge in terms of whether these networks have actually fulfilled their potential to facilitate interdisciplinary collaboration in the form of a learning community and how to study this. This paper aims to take a key step to fill that gap, by examining whether an intentionally designed network can provide the opportunity and means for interdisciplinary learning over time. A secondary aim of this paper was to demonstrate the application of social network research to studying learning communities in the field of LHS.

Methods

Study setting and participants

The Australian Genomic Health Alliance¹⁴ (hereafter, Australian Genomics) is a nationwide network of clinicians, scientists, researchers and community representatives from varied sites, medical specialties and contexts. Genomics is a fast-moving field where interpretation of findings relies heavily on keeping up to date with research advances and interdisciplinary discussions,¹⁵ making this an appropriate field for investigating a macrolevel LHS.

The stated aim of Australian Genomics is to prepare Australia for the integration of genomic medicine into routine healthcare.¹⁴ It was intentionally designed as a national and multidisciplinary entity with programmes of activity covering all aspects of genomic medicine: from data storage and sharing issues, to building the evidence base of efficacy, clinical and cost effectiveness, and implementation, policy and ethics.¹⁴ Established in 2016, Australian Genomics invited health professionals and consumers from all states and territories in Australia to take part in a national collaborative endeavour to move genomic medicine into usual practice. Funded by the National Health and Medical Research Council, Australian Genomics supported Flagship teams (chosen through competitive grant proposals) to build the evidence base of clinical utility and effectiveness across a range of conditions (eg, renal, cardiology, intellectual disability, cancer and neonatal syndromes). Strategies for facilitating continuous learning in the network included forming working groups to bring disparate disciplines together to design tools such as consumer resources explaining genomic testing, data sharing and management platforms. Communication between Australian Genomics members was facilitated by newsletters, frequent updates on their website, email communications and presentations from the various working groups and Flagships. More information about Australian Genomics can be found in our previous publications.^{15 16}

Study participants were all members of Australian Genomics as of 1 May 2018. Members had all formally joined, received newsletters, updates and announcements and were asked to report annually on associated research outputs. Members were further defined by their participation in a flagship project, programme working group, governance body (eg, National Steering Committee, Community Advisory Group), or being operational staff (eg, project officers, manager). Participants were recruited via emails sent by Australian Genomics on behalf of the researchers, after the research had been publicised in meetings and official communications in preceding weeks. Potential participants were informed that no negative influence was imposed on any individual that chose not to participate in the study.

Social network research

Social network research is a way to explore relationships such as communication pathways, or social structures such as silos, within a specified group of people or sites.^{17 18} There are many different options when designing and conducting social network research.¹⁹ For example, research can be targeted to egonets (personal relationship networks around one agent) or whole networks (by defining a boundary and examining all personal links within that system). Network research can be cross-sectional or longitudinal and examine network characteristics at multiple levels (unipartite, multiplex, bipartite). Data collection can be achieved through name interpreters, name generators and position generators, using qualitative, quantitative, or mixed-method formats.^{19 20} A common method used in health services research, especially when working with large numbers,¹⁹ is a social network survey using a name interpreter roster. The survey includes a ‘roster’ of names of people considered to be members of the network, for participants to look through and indicate with whom they have the described relationship (eg, collaborate).²¹ A glossary of social network terms is shown in table 1.

Table 1

Glossary of complexity and social network terms

Term	Definition
Centralisation	Extent to which the network is focused around one or few central people. Low centralisation indicates a more even distribution of ties.
Centrality	A measure to identify which players have the most interaction with others, that is, the most prominent, ‘key’ players. Centrality of 1 indicates that the actor is interacting with all members of the network.
Density	The proportion of ties found across a network per the number of possible ties.
Indegree	A measure of influence. Number of ties directed to a node, that is, the number of times a particular individual is nominated by others as having that relationship with them.
Isolates	Agents or individuals with no links to others in the network.
Nodes	Agents or individuals. Depicted as dots or small circles in sociograms.
Outdegree	A measure of connectedness. Number of ties a particular node directs to other nodes, that is, the number of other people a particular individual nominates as having that relationship to them.
Silo	A group of people characterised by their limited interaction with others.
Social Network	A system of social interactions and personal relationships with interactions between them.
Sociogram	A graphical depiction of the relationship data in a social network study collected from individuals and then collated. Based on graph theory, parameters can be computed from the aggregated data.
Ties	The relationship of interest in a social network study. Two nodes are said to be tied if one or both acknowledge the relationship. Depicted as a line between nodes.

Data collection

An online questionnaire was designed by an expert panel of health services researchers with network and implementation expertise, and key informants from Australian Genomics. The first part of the questionnaire asked social network questions about respondents’ collaborative links to define the Australian Genomics’ learning community and were told that we were trying to capture who was ‘part of their genomics learning community’. The question read: ‘Please work your way down the list [of names] and indicate who you work with on Australian Genomics projects. By ‘work with’ we mean in the context of genomics—shared care of patients, worked in the same lab, been involved in research together, participated in a working group together, had a phone call about Australian Genomics etc’.

In the second part of the survey, participants were asked three questions about how learning occurs in the network. Specifically, to what extent: (1) formal sources (formal course, journal articles or conference presentations, in-service symposiums or teaching sessions), (2) informal learning (‘hands on learning’, observation, making decisions collectively, trial and error), and (3) group influence (governance, patients/parents/families, national steering committee, other flagships, other activities in the same or other programmes) played a role in learning. Each option was rated on a five-point Likert scale from ‘not at all’ to ‘to a large extent’ with an option for ‘not applicable’.

In the last part of the survey, respondents were asked open-ended questions. This paper particularly focuses on the most general question: ‘Do you have any comments you would like to make?’ The question included specific prompts: ‘Stories about collective learning, or your experience of working within a genomic network?’ The questionnaire was created on the survey platform Qualtrics. A unique, secure link to the online questionnaire, was distributed to members of Australian Genomics during May 2018 (capturing network data in 2018 and retrospective data from 2016) and again in June 2019. The methods were consistent between both rounds of data collection. The only change made to the questionnaire was in regards to the data collection roster tool that was amended to accurately represent the organisational changes of Australian Genomics. Namely, two new flagship projects commenced and the subgroups of programme 2 were combined to one overarching programme in 2019.

Analysis

Social network data were analysed and visually produced using Gephi V.0.9.2 (CP, JL).²² Networks were analysed over three time points (2016, 2018, 2019). Network parameters were used to assess network growth and change: number of nodes, number of ties, number of isolates, indegree, outdegree and density. Density was computed in Gephi. Missing tie data (from non-respondents) was not imputed. Key players were determined based on indegree and outdegree scores.

Likert style questions were analysed using descriptive statistics, paired t-tests and one-way analysis of variance (ANOVA) in SPSS V.27. Open responses to survey questions were analysed in NVivo V.12. As answers were relatively brief, themes were coded inductively based on key words (JL, CP). Responses were coded into five themes, developed inductively: learning community; compliments and personal impacts; project size; operational factors (ie, administration, coordination and communication within Australian Genomics); and strategic factors (ie, aims and objectives, governance or strategies).

Patient and public involvement

No patients or members of the public were actively involved in the design, conduct, reporting, or dissemination plans of our research.

Results

Network data

Characteristics of survey respondents are presented in table 2. In 2019, 183 of 439 (41.7%) Australian Genomics members completed the network census. Four members did not consent, and 15 responses were not included due to incomplete data. This was lower than the 2018 survey response rate (n=222/384, 57.8%). Nodes were created for all the network members (total invited) and all reported ties were included (to non-respondents as well as respondents). Since 2016, the Australian Genomic learning community has grown from 186 to 384 in 2018 and 439 members in 2019; table 3 summarises parameters for this network over time. Data are visually presented in figure 2A,B,C showing that the network has grown in number of nodes and ties since 2016, although a reduction in density was found between 2018 and 2019. Missing tie data from non-respondents were not imputed in the density calculation but we considered the effect of this to be acceptably small. We were confident that the most engaged in the network, who also had the highest and most influential outdegree had completed the survey. Visualisation shows that the network maintained interdisciplinary links over time (ie, there are no silos based on component analysis).

Table 2

Characteristics of survey respondents

Parameter	2016–2018		2019
	Total invited n=384 n (%)	Respondents n=222 n (%)	Total invited n=439 n (%)	Respondents n=183 n (%)
Females	202 (52.6%)	122 (55.0.4%)	230 (52.3%)	120 (65.6%)
Males	182 (47.4%)	100 (45.0%)	209 (47.6%)	63 (34.4%)
Medical specialists	73 (19.0%)	25 (11.3%)	103 (23.4%)	74 (40.4%)
Genetic specialists	94 (24.5%)	71 (32.0%)	111 (25.2%)	91 (49.7%)
Medical scientists	100 (26.0%)	52 (23.4%)	98 (22.3%)	89 (48.6%)
Researcher	42 (10.9%)	27 (12.2%)	39 (8.9%)	34 (18.6%)
Other	75 (19.5%)	47 (21.2%)	88 (20.0%)	60 (32.8%)

‘Researcher’ include biomedical, health services, and sociological researchers; ‘Other’ includes Operational staff, students, consumers.

Table 3

Comparison of Australian genomics learning community over time

Parameter	2016	2018	2019
Number of nodes	186	384	439
Number of respondents	222	222	183
Number of ties	2925	6381	6875
Number of isolates	27	5	24
Highest indegree	44	91	109
Highest outdegree	87	354	399
Density	0.020	0.043	0.036

Figure 2. Sociograms. Nodes are Australian Genomic members and size of node is indicative of indegree (the bigger the node the more highly nominated). Australian Genomics learning community. Colours show the respondents’ groups (seven groups with the most ties of a total of 38 group). Operations, KidGen Renal Genetics, Acute Care Genomic Testing, Genetic Immunology, Cardiovascular Genetic Disorders, National Steering Committee, Acute Lymphoblastic Leukaemia. (A. 2016 (knew-before); B. 2018; C. 2019).

Since the collection of the first round of survey data (2016 and 2018) until the second round (2019), 34 members had left the network and 90 new members had joined. Most recent (2019) network data revealed that new members joining the Australian Genomic learning community have the potential to achieve focal positions in the network (figure 3). Indeed, some new members yielded relatively high scores of indegree (72—Administrative Assistant) and outdegree (80—Data Coordinator). However, not all new members were as integrated in the network; 12/24 (50.0%) of isolates were new members who joined the network in 2019. Key players remained mostly consistent over time. Most key players were nationally based or from Victoria (where the national staff of Australian Genomics are physically based). Operations and management staff maintained key player positions in the network over time; specifically the Australian Genomics Manager (national) and Project Officers (national) were key, as well as a clinical geneticist involved in several flagships and working groups from Victoria. Results also revealed international collaborative links to agents outside of Australia; 412 external collaborators were nominated by participants in 2018 and 363 in 2019.

Figure 3. Australian Genomics learning community: comparison of new and old members 2019 (the bigger the node the more highly nominated). Colours indicate member. existing member, new to the network in 2019.

How learning occurs

Formal learning, informal learning and group influence all contributed to learning in the network (figure 4). Responses to Likert style questions revealed that informal learning (2018, M=3.79; 2019, M=3.75) was the most influential learning method for genomic practice compared with formal learning (2018, M=3.14; 2019, M=3.11) and group influence (2018, M=3.12; 2019, M=3.20). Across the two surveys (2018, 2019), ratings for informal learning were significantly higher than either formal learning, t(228)=9.34, p<0.001, or group influence, t(267)=12.33, p<0.001. Specifically, the two most influential modes of informal learning were ‘hands on learning’ (2018, M=4.16; 2019, M=4.09) and making decisions collectively (2018, M=4.05; 2019, M=4.19). Across the two surveys (2018, 2019), there was no significant difference between formal learning and group influence, t(205)=−1.66, p=0.10. Additional ANOVA tests indicated that there were no significant differences between the different learning modes or shifts in learning modes over time.

Figure 4. Influential learning methods for genomic practices. F, Formal, I, Informal, GI, Group Influence.

Open responses about the network: a learning community

The two most frequent themes were around the idea of Australian Genomics as a learning community, and personal impacts (see online supplemental appendix 1, e.g., quotes). For the purposes of this study, we focus on open-response data coded as ‘Learning Community’. Participants commented on the nature of continuous learning and how the network was integral to navigate this: ‘This is an ever-evolving field and what we understand today may be different tomorrow. The only way to stay on top of this evolution is to collaborate effectively with clinicians and researchers’ [P201 genetic specialist]. Participants also revealed structural issues in the network that may reduce the learning ability: ‘Our program has not had too many opportunities to work with members of other programs and flagships’ [P354 genetic specialist] due to the sheer size of the network: ‘so large that at times it is hard to keep up with who is doing what, and where’ [P3 researcher]. For the most part, the network was seen as a supportive and positive factor to foster continuous learning within and between groups and disciplines: ‘It’s great to have a working group I can regularly refer to for input and advice’ [P97 researcher]; ‘Often links between different professional groups are hard to initiate and I think Australian Genomics has played a great role in helping to bring a wider genomics community together’ [P99 genetic specialist].

Discussion

Purposely designed networks, such as Australian Genomics, can bring us closer to the vision of an LHS by fostering continuous interdisciplinary learning. This case study of Australian Genomics shows that creating a network with a purpose of continuous learning (a key behavioural component of an LHS) fosters continuous learning over time. The Australian Genomics community has grown over 4 years from 186 members in 2016 to 439 in 2019. According to members, continuous learning occurs formally, informally and based on group influence, but is most influential when informal through ‘hands on learning’ or collective decision-making. Overall findings build on our previous work¹⁵ showing that the Australian Genomics learning community is interdisciplinary and collaboration and learning occurs across traditionally siloed groups (eg, across medical specialties each now using genomics). Investment in strategies to structure working groups and facilitate a national, interdisciplinary approach are clearly supporting these outcomes. Network members highlighted the benefits of the network in terms of ability to seek advice to support their continuous learning in the ever-evolving field of clinical genomics.

In healthcare, whether in the field of clinical genomics or more generally, continuous learning and integration of new knowledge into the system is essential to keep up to date with the best practices. Given the fast-paced generation of medical knowledge and new research, the collaborative and learning potential of having an established system of communication and knowledge transfer (such as a network) far supersedes the learning potential of any one individual. Indeed, there are many explanations and models of how groups, and the social processes between members can deliver better outcomes than people working alone. For example, multidisciplinary teams in cancer care ensure all the expertise needed to comprehensively assess, plan and coordinate care for a patient is available at the same time.²³ Innovation, a valuable outcome for many organisations, has been conceptualised as the novel recombination of diverse knowledge and experience held by different individuals.^24–26 Translational research networks bridge the gap between scientists undertaking basic research and clinicians delivering care by providing a structure that supports collaboration.²⁷

This study demonstrates the application of social network research as a key tool to visualise and quantitatively analyse social-based learning in the LHS field. Social network questions such as ‘who is part of your genomic learning community’ elicit a wealth of data as respondents nominate people from other states, medical specialties or research areas. Consumers from the advisory board were also frequently nominated. Other data collection methods, such as observations of social learning during meetings, cannot expect to show this richness. While each social network survey is a snapshot, the dynamic nature of LHSs can be captured through longitudinal social network studies and resampling. However, a noted limitation of these methods is that data can be time consuming to collect and survey fatigue can limit participation. Nevertheless, the resulting sociograms are compelling graphics that tell a clear story and can be used as an intervention to diagnose potential problematic network areas that have not integrated.

This study is the first to highlight that creating a network with the purpose of continuous learning can foster interdisciplinary learning over time, contributing to the vision of an LHS. Other benefits of having a network in an LHS, as noted in past literature include: establishing a national repository of data for meaningful centralised analysis (and in the case of Australian Genomics, tapping into international repositories); interoperability between systems facilitated by open technological standards; and, developing local and national systems which can synergise to feedback data to organisations.²⁸ The next step in this field is to evaluate whether establishing a learning network fulfils its aims as a continuous learning community. That is, does creating a purposely designed network have an impact on workforce outcomes (learning and development, professional well-being) and patient outcomes?

Strengths and limitations

A limitation is that this study only assessed one component (continuous learning) of an LHS. While focusing the consideration of an LHS to one component without considering the others (such as science and informatics) may have limited the scope of our findings, focusing on one factor enabled an in-depth exploration of how collaborative learning has occurred over time. Another limitation is the issue of a suboptimal response rate and missing data in the whole network social network design. Whole network studies are most affected by missing data when key players do not respond; however, in this case, most operational staff and project leads responded, and therefore we have no major concern over the accuracy of the network data.²⁹ Another limitation of the social network survey approach is that the resultant network is a snapshot of a network that is inherently dynamic. Lastly, the data collection for the year 2016 was retrospectively captured in 2018, leaving room for potential recall bias. To the best of our knowledge, this was the first study to show that learning in a macrolevel LHS occurs across disciplines and specialties, as is idealised. An additional strength of this study is that we demonstrated that intentionally designed networks provide the opportunity and means for interdisciplinary learning between diverse agents over time and demonstrate the application of social network research to the LHS field.

Conclusion

This study identified advantages of intentionally designed networks to foster learning within a macrolevel LHS. In the case of the Australian Genomics learning community, we found evidence that an intentionally designed network can provide the opportunity and means for interdisciplinary learning, that was enduring over the 4 years of data collection. This study shows that continuous learning is most influential when informal, through ‘hands on learning’ or collective decision-making. While we cannot claim cause and effect from our findings, it seems clear that the structure of the network is associated with the LHS outcomes. This study also demonstrated the potential for social network research as a practical and appropriate approach to studying learning communities over time, as a component of the LHS.

Data availability statement

Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics statements

Patient consent for publication

Consent obtained directly from patient(s)

Ethics approval

This study was approved by the Macquarie University Human Research Ethics Committee (ID: 5201701186) and was endorsed by the Australian Genomic Health Alliance National Steering Committee. Participants gave informed consent to participate in the study before taking part.

Footnote

Twitter @JanetCLong, @zeyadmahmoud

Contributors LAE, CP, RL and JCL analysed the data. LAE and CP drafted the manuscript with contribution from ZM, JCL, RL, GD and JB. LAE accepts full responsibility for the conduct of the study.

Funding JB reports funding from the National Health and Medical Research Council: NHMRC Partnership Centre in Health System Sustainability (Grant ID 9100002), NHMRC Investigator Grant (Grant ID 1176620) and NHMRC Genomics Health Alliance 2.0 (Gant ID 2000001).

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Institute of Medicine. Best care at lower cost: the path to continuously learning health care in America. Washington (DC): Institute of Medicine, 2013.

² Institute of Medicine. The learning healthcare system: workshop summary. In: Olsen L, Aisner D, McGinnis JM, eds. Washington (DC): Institute of Medicine, 2007.

³ Institute of Medicine. Digital infrastructure for the learning health system: the foundation for continuous improvement in health and health care. Washington (DC): Institute of Medicine, 2011.

⁴ Budrionis A, Bellika JG. The learning healthcare system: where are we now? A systematic review. J Biomed Inform 2016; 64: 87–92. doi:10.1016/j.jbi.2016.09.018 http://www.ncbi.nlm.nih.gov/pubmed/27693565

⁵ Platt JE, Raj M, Wienroth M. An analysis of the learning health system in its first decade in practice: scoping review. J Med Internet Res 2020; 22: e17026. doi:10.2196/17026 http://www.ncbi.nlm.nih.gov/pubmed/32191214

⁶ SK-l ZY, Vedovi A, Ellis LA. Mapping the learning health system: a scoping review of current evidence. Sydney, Australia: Australian Institute of Health Innovation, and the NHRMC Partnership Centre for Health System Sustainability, 2020.

⁷ Braithwaite J, Glasziou P, Westbrook J. The three numbers you need to know about healthcare: the 60-30-10 challenge. BMC Med 2020; 18: 1–8. doi:10.1186/s12916-020-01563-4

⁸ Pomare C, Mahmoud Z, Vedovi A, et al. Learning health systems: a review of key topic areas and bibliometric trends. Learn Health Syst 2022; 6: e10265. doi:10.1002/lrh2.10265

⁹ Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high-functioning learning health system. J Am Med Inform Assoc 2015; 22: 43–50. doi:10.1136/amiajnl-2014-002977 http://www.ncbi.nlm.nih.gov/pubmed/25342177

¹⁰ Ellis LA, Sarkies M, Churruca K, et al. The science of learning health systems: scoping review of empirical research. JMIR Med Inform 2022; 10: e34907. doi:10.2196/34907 http://www.ncbi.nlm.nih.gov/pubmed/35195529

¹¹ Pomare C, Mahmoud Z, Vedovi A, et al. Learning health systems: a review of key topic areas and bibliometric trends. Learn Health Syst 2022; 6: e10265. doi:10.1002/lrh2.10265 http://www.ncbi.nlm.nih.gov/pubmed/35036549

¹² Britto MT, Fuller SC, Kaplan HC, et al. Using a network organisational architecture to support the development of learning healthcare systems. BMJ Qual Saf 2018; 27: 937–46. doi:10.1136/bmjqs-2017-007219 http://www.ncbi.nlm.nih.gov/pubmed/29438072

¹³ Lorts A, Smyth L, Gajarski RJ, et al. The creation of a pediatric health care learning network: the action quality improvement collaborative. Asaio J 2020; 66: 441–6. doi:10.1097/MAT.0000000000001133 http://www.ncbi.nlm.nih.gov/pubmed/32224822

¹⁴ Australian Genomic Health Alliance. Australian genomic health alliance 2018. Available: https://www.australiangenomics.org.au/ [Accessed July 2018 ].

¹⁵ Long JC, Pomare C, Best S, et al. Building a learning community of Australian clinical genomics: a social network study of the Australian genomic health alliance. BMC Med 2019; 17: 1–13. doi:10.1186/s12916-019-1274-0

¹⁶ Stark Z, Boughtwood T, Phillips P, et al. Australian genomics: a federated model for integrating genomics into healthcare. Am J Hum Genet 2019; 105: 7–14. doi:10.1016/j.ajhg.2019.06.003 http://www.ncbi.nlm.nih.gov/pubmed/31271757

¹⁷ Wasserman S, Faust K. Social network analysis. Cambridge: Cambridge University Press, 1994.

¹⁸ Robins G. Doing social network research: network-based research design for social scientists. Sage, 2015.

¹⁹ Pomare C, Long JC, Churruca K. Social network research in health care settings: design and data collection. Social Networks 2019; 69. doi:10.1016/j.socnet.2019.11.004

²⁰ Long JC, Bishop S. Social network research. Springer, 2018.

²¹ Pomare C, Long JC, Ellis LA, et al. Interprofessional collaboration in mental health settings: a social network analysis. J Interprof Care 2019; 33: 1–7. doi:10.1080/13561820.2018.1544550 http://www.ncbi.nlm.nih.gov/pubmed/30411988

²² San Jose. Gephi: an open source software for exploring and manipulating networks. International AAAI conference on Weblogs and social media. California, 2009.

²³ Basta YL, Bolle S, Fockens P, et al. The value of multidisciplinary team meetings for patients with gastrointestinal malignancies: a systematic review. Ann Surg Oncol 2017; 24: 2669–78. doi:10.1245/s10434-017-5833-3 http://www.ncbi.nlm.nih.gov/pubmed/28337661

²⁴ Hargadon A, Sutton RI. Technology brokering and innovation in a product development firm. Adm Sci Q 1997; 42: 716–49. doi:10.2307/2393655

²⁵ Schumpeter J. The theory of economic development. Cambridge, M.A Harvard University Press; 1934.

²⁶ Burt RS. Structural holes and good ideas. Am J Sociol 2004; 110: 349–99. doi:10.1086/421787

²⁷ Long JC, Hibbert P, Braithwaite J. Structuring successful collaboration: a longitudinal social network analysis of a translational research network. Implement Sci 2016; 11: 19. doi:10.1186/s13012-016-0381-y http://www.ncbi.nlm.nih.gov/pubmed/26864452

²⁸ Cresswell KM, Smith P, Swainson C. Establishing data-intensive learning health systems: an interdisciplinary exploration of the planned introduction of hospital electronic prescribing and medicines administration systems in Scotland. BMJ Health & Care Informatics 2016; 23. doi:10.14236/jhi.v23i3.842

²⁹ Borgatti SP, Carley KM, Krackhardt D. On the robustness of centrality measures under conditions of imperfect data. Soc Networks 2006; 28: 124–36. doi:10.1016/j.socnet.2005.05.001