Co-Produce, Co-Design, Co-Create, or Co-Construct—Who Does It and How Is It Done in Chronic Disease Prevention? A Scoping Review

Abstract

Co-production in health literature has increased in recent years. Despite mounting interest, numerous terms are used to describe co-production. There is confusion regarding its use in health promotion and little evidence and guidance for using co-produced chronic disease prevention interventions in the general population. We conducted a scoping review to examine the research literature using co-production to develop and evaluate chronic disease prevention programs. We searched four electronic databases for articles using co-production for health behaviour change in smoking, physical activity, diet, and/or weight management. In 71 articles that reported using co-production, co-design, co-create, co-develop, and co-construct, these terms were used interchangeably to refer to a participatory process involving researchers, stakeholders, and end users of interventions. Overall, studies used co-production as a formative research process, including focus groups and interviews. Co-produced health promotion interventions were generally not well described or robustly evaluated, and the literature did not show whether co-produced interventions achieved better outcomes than those that were not. Uniform agreement on the meanings of these words would avoid confusion about their use, facilitating the development of a co-production framework for health promotion interventions. Doing so would allow practitioners and researchers to develop a shared understanding of the co-production process and how best to evaluate co-produced interventions.

1. Introduction

In Australia, as in many other high-income countries, chronic diseases place a significant and persistent burden on the community, with the social and economic consequences having a detrimental effect on an individual’s quality of life [1]. Much of the burden caused by chronic disease is preventable, with modifiable risk factors including tobacco use, overweight and obesity, physical inactivity, and unhealthy dietary behaviours [2]. Governments are often the lead agency for the development and the implementation of health promotion strategies and programs to prevent health behaviour-related chronic disease at the population level, as well as research into their effectiveness as a way of promoting healthy choices, preventing disease, and keeping people out of hospital [3,4]. ‘Co-production’ is a mechanism whereby ‘stakeholders’ (identified as including end users or intervention target audience, health researchers, academics, policy and practice partners, decision makers, and funding representatives) can collaborate, generate relevant knowledge, and apply it to practice [5,6,7]. It involves key stakeholders in the development of health-related interventions based on the premise that involving the target audience or intermediaries in the design and implementation is likely to have a positive impact on the effectiveness of the service or program [8].

The number of publications that identify the use of co-production in health literature has risen markedly in recent years. While it has been suggested that the growth has been approximately 25% per year from 2004 to 2019 [9], a keyword search for ‘health’ and ‘co-production’ using the Web of Science database indicates that the increase appears to be even steeper over the past ten years. However, little is known about how co-production facilitates the development of effective programs. Clarke et al. [10] found that evaluations of the outcomes of co-produced interventions designed to improve the quality of acute health care services lacked rigour, particularly when assessing clinical and service outcomes and cost-effectiveness. It appears that co-production is difficult to evaluate and cannot be evaluated by the standard evidence hierarchy in the evidence-based practice movement [11]. While some evaluation frameworks exist, effectiveness is mostly demonstrated through the use of case studies [12]. Despite mounting interest, there is limited evidence and guidance for co-produced chronic disease prevention interventions in the general population [13], and to our knowledge, there are no reviews.

Another factor that may hinder the advancement of the field is the heterogeneity of the terms used to describe such a process in the health setting (e.g., co-produce, co-design, co-create) [14], a problem also reported in the public administration setting [15]. That literature has sought to clarify the concept by considering who is involved in co-production (and at what level: individual, group, or collective), what occurs in co-production, and when it occurs [16]. Similarly, untangling the confusion around co-production in health and chronic disease prevention settings is important. There is a need, and interest, to identify where the literature could be enhanced using a shared understanding of what co-production is (and is not), what other terms might be used to mean the same thing, who the key players in co-production are, and what about co-production might make an intervention effective [17,18].

The purpose of this study, therefore, was to undertake a systematic scoping review to determine the size and extent of available research literature on the use of co-production in the development and evaluation of primary research studies of health behaviour change programs for the prevention of chronic disease. In particular, we sought to answer (a) how co-production is used in the development and evaluation of chronic disease prevention programs, (b) who is usually involved in the co-production of the development and evaluation of chronic disease prevention programs, and (c) whether and how the literature reports on the evaluation of co-produced chronic disease prevention programs.

2. Materials and Methods

We conducted a scoping review to identify the nature and extent of co-produced interventions addressing chronic disease prevention [19]. A scoping review was chosen because its purpose is to identify the types of evidence in a research field, clarify key concepts, explore how research is conducted on a topic, and identify gaps in the literature [20]. The methodology was guided by an established framework for scoping studies [21,22]. The Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews (PRISMA-ScR) checklist was used to ensure that this scoping review achieved quality standards of practice and reproducibility [23].

The development of the search strategy was iterative. First, we conducted a preliminary search of the literature, which included the term ‘co-production’ to identify other terms related to co-production that were commonly used in the literature [6,14]. These terms were: co-design, co-create, co-construct, partnership, and collaboration. A further literature search was conducted to identify a sample of articles using ‘partnership’ or ‘collaboration’ in the development and evaluation of chronic disease prevention programs. Two authors (B.M. and B.O’H.) screened the full text of these papers to determine how partnership (n = 16) and collaboration (n = 12) were used in these studies. We found articles using the terms ‘collaboration’ or ‘partnership’ did so to loosely refer to intersectoral (including academic, government, and financial) relationships. The studies screened did not define or describe what was meant by partnership or collaboration in sufficient detail in relation to the development or evaluation of programs with a chronic disease prevention focus for these terms to be included in this study. This observation is supported by Johnston and Finegood [24], who acknowledge the ambiguous and interchangeable use of the term partnership with other terms including collaboration. Therefore, ‘partnership’ and ‘collaboration’ were excluded from the final search strategy as they were concepts considered broader than co-production.

We systematically searched electronic databases for peer-reviewed literature (Medline via Ovid, PsycINFO via Ovid, Cinahl, Scopus, and PubMed) for articles using a co-production, co-design, co-creation, and co-construction approach to achieve a lifestyle-related health behaviour change [6,14]. The search was conducted in May 2020 and updated in March 2021. Search terms included a combination of medical subject headings (MeSH) terms and keywords as outlined in Table S1. Reference lists of all included studies and relevant interventions known to the authors were searched for additional studies. We included English, peer-reviewed articles with studies reporting protocols and/or outcomes of interventions (primary and secondary prevention) using a co-production, co-design, co-creation, and co-construction approach in any health promotion setting. The outcomes of interest were chronic disease prevention-related behaviours outcomes such as smoking, physical activity, diet, and/or weight management. The populations of interest included individuals, communities, or populations at risk of developing health behaviour-related chronic disease. We did not limit studies by research design or publication year. Studies of interventions with a clinical orientation (e.g., those targeting service delivery, rehabilitation, or medication adherence) were excluded.

Search results were combined in EndNote X9.3.3, and duplicate references were removed. Three authors (B.M., B.O’H., and L.C.) independently scanned titles and abstracts to determine inclusion eligibility. The full text of eligible papers was independently reviewed by B.M., B.O’H., and L.C. according to the pre-determined inclusion criteria outlined above, with discrepancies resolved by discussion. Study characteristics extracted included how study authors defined the ‘co-words’ used, who was involved in the co-production, the process and impact measures to evaluate interventions using co-production, and the sections of the paper where co-words were mentioned. Data were further tabulated according to the purpose of the study, the prevention focus, the target population, and the collaborative technique used in relation to co-production (Table S2).

3. Results

3.1. Study Selection

Database searches identified 589 publications, and an additional two publications were identified by citation searching. Following duplicate removal and title and abstract screening, 117 full-text articles were reviewed for inclusion eligibility, resulting in the inclusion of 71 articles (64 unique studies and 7 reviews) (Figure 1).

3.2. Study Characteristics

3.2.1. Publication Country

Articles that address co-production were from a range of countries: the U.K. (England [25,26,27,28,29,30,31,32,33], Scotland [34,35,36] and Wales [13,37]), the European Union [38] (The Netherlands [39,40,41,42,43,44,45,46], Belgium [47,48], Denmark [49,50], Ireland [51], Italy [52,53], Estonia [54], Sweden [55], Greece [56], France [57] and Spain, Italy and U.K. [58,59]), Australia [60,61,62,63,64,65,66,67,68,69,70,71,72], USA [73,74,75,76,77], Canada [78,79], New Zealand [80,81], Brazil [82] and Lebanon [83]. The studies included in review articles had an global focus, and authors were from Australia [84,85,86], New Zealand [87], the U.K. [88,89] and Germany, Switzerland, Australia, The Netherlands, USA and Canada [90]. The review included seven protocol papers [32,34,46,48,68,77,81], seven review papers [84,85,86,87,88,89,90] and the remaining described intervention development and/or the impact of the intervention.

3.2.2. Prevention Focus

The prevention focus of articles included health promotion and healthy lifestyle [40,41,42,43,65,73,81,84,85,87,90,91,92], health policy and chronic disease prevention [32,62,66,88], obesity prevention [33,34,35,49,52,58,59,61,77,80,89,93], physical activity [25,30,37,38,39,47,48,51,53,54,79,82,83], physical inactivity [44], sedentary behaviour or sitting [27,31,64], healthy eating or dietary behaviour change [26,46,57,68,69,71,74,75,78], smoking cessation or prevention [13,45,56,60,70] and alcohol abuse prevention [50,63,94] (Table 1). The target populations in terms of general age-groups included children [33,34,35,37,39,42,43,46,77,83,93], adolescents [40,41,47,51,52,58,61,84,85,86,91,92,94] or both [25,54,65], families of preschool children [49], mothers of infants and children [73,75], fathers and their children [48], young adults [45,68,78], adults [28,29,53], adults with intellectual disability [88], and older adults [30,36,38,44,55,79]. More specifically, food outlet managers or consumers [26,69,72], Aboriginal or First Nation people and communities [13,60,63,67,70,71,74,76,80,81], ethnic [66] or socioeconomically disadvantaged communities [32,33,57,73], and desk-based workers [27,31,64] and commercial stakeholders [56] were also the focus.

3.2.3. ‘Co-Word’

Articles used five different ‘co-‘ words: co-design, co-create, co-produce, co-construct and co-develop. While these words appeared throughout the text of the articles, they were used at least once in the abstracts of 65 articles and in the discussion and conclusions of 57 articles. Five articles used the word co-design in only the abstract [26,60,66,74,77]. As a single term, co-design (n = 25, 34.7%) [26,53,55,58,59,60,61,63,64,65,66,67,71,74,76,77,78,79,80,81,84,86,87,91,93], co-create (n = 17, 24%) [27,33,36,38,39,40,41,45,46,47,48,50,51,56,73,75,85] and co-produce (n = 8, 11.1%) [13,29,31,34,35,37,89,90] were used most frequently. Co-construct (n = 2, 2.8%) [44,57] and co-develop (n = 1, 1.4%) [25] were the least prevalent. In a quarter of the articles (n = 18, 25%) two or more ‘co-‘ words were used interchangeably in the same paper, most frequently including co-design and co-create [42,49,69,72,82,92], co-design and co-develop [30,70], and co-create and co-design [54,68]. For consistency, from this point the words ‘co-production’ or ‘co-produce’ will be used to refer to all ‘co-‘ words (co-design, co-create, co-develop and co-construct).

3.3. The Operationalisation of Co-Production in the Development and Evaluation of Chronic Disease Prevention Interventions

Co-design was reported in 39 articles (including 4 reviews) [25,26,30,32,42,49,52,53,54,55,58,59,60,61,63,64,65,66,67,68,69,70,71,72,74,76,77,78,79,80,81,82,83,84,86,87,88,91,92,93]. Of these, seven explicitly defined co-design as it was used in the development of a chronic disease prevention intervention [53,63,69,72,82,84,87], 28 described how they used co-design but did not define it explicitly [25,26,32,42,49,52,54,55,58,59,60,61,65,66,70,71,74,76,77,78,79,80,81,83,88,91,92,93] and four did not specify what was meant by co-design [30,64,67,86]. All articles that defined co-design, plus an additional article with a co-creation focus [68], identified that co-design aligned with a participatory design approach, in which end users or stakeholders of intervention are engaged in the research process. As one of the papers stated, the process of co-design is illustrated as a ‘golden thread’ running through all stages of public health research [84], enabling the contributions of end users to be incorporated from intervention development and testing to implementation and dissemination [63,82,87]. In particular, the aim is to empower stakeholders as part of the design process by recognising their expertise in their own experiences [53,69]. Intervention co-design was described in these studies as an iterative and creative collaboration or partnership between end users and relevant stakeholders and intervention designers [53,63,84,87]. While not all articles explicitly defined what they meant by co-design, collaboration between multiple stakeholders as part of the co-design process was described in the methods in the majority of papers reporting co-design [25,53,54,55,58,60,61,65,66,74,79,80,81,82,83,84,87,92]. In terms of stakeholders involved, in the studies we identified intervention development occurred in consultation with the community or industry [60,65,66,70,71,74,76] and end users [42,43,55,58,61,79,91,92].

Co-creation was included in 23 articles (22 unique studies and 1 review) [27,36,38,39,40,41,42,43,45,46,47,48,49,50,51,56,62,68,72,73,75,85,94]. Six included a definition of co-creation [38,45,48,68,72,85], 12 described how they incorporated co-creation [27,36,39,40,41,47,49,50,51,56,73,75], and despite using the term within their paper, five did not clearly outline what was meant by co-creation in the study [42,43,46,62,94]. In those papers that did define it, co-creation was defined as an active process between people with shared goals but different expertise and skills, by which stakeholders were enabled to be directly involved in the generation of an intervention or solution [38,45,48,68,72,85]. There were similarities in how co-creation was defined to the definitions of co-design described above. Namely, co-creation is described as having developed from participatory design [85] and the collaborative engagement of all stakeholders, including end users, throughout the process of developing and implementing an intervention [38,48,68,72,85]. Two articles described the inclusion of co-creation in the development of behaviour change interventions as (a) reducing barriers to change [56] and (b) providing insights into motivation for change [51].

Co-production was reported in 16 articles (13 unique studies and 3 reviews) [13,25,28,29,31,32,33,34,35,37,52,62,88,89,90,94]. Two studies provided an explicit definition of co-production [33,37], eight described the process by which co-production was used to develop or evaluated their intervention [13,28,29,32,34,35,52,90], and six did not provide details on what they meant by co-production [25,31,62,88,89,94]. Similar to co-design and co-creation, co-production was defined as involving the target audience in the design and implementation of an intervention [37]. The process of co-production, according to these articles, incorporated the implementation, stakeholder, and participant contexts into the intervention development, implementation, and evaluation [32]. As with co-design and co-creation, co-production was described as using participatory approaches to involve participants and stakeholders in an equal and reciprocal relationship [35,52] for the iterative development of an intervention [13,28,29]. The authors felt that such a process gave rise to services that meaningfully met the needs of individuals and communities, and represented an engaging process to achieve behaviour change among end users [33].

Co-construction was used in three unique studies [44,57,62]. None defined co-construction but described the process as involving collaboration with stakeholders and end users through all stages of intervention development and evaluation [44,57,62].

A variety of co-production techniques were described, but there was no clear pattern of use according to which ‘co’-word was used (Table 2). Group sessions, focus groups and discussions were used in 31 unique studies (34 articles) [13,26,28,29,32,33,34,35,37,39,40,41,42,43,44,46,47,48,51,53,59,62,67,73,74,76,77,78,80,81,83,91,92,93], workshops with stakeholders were used in 20 studies [25,30,33,36,38,49,50,55,57,58,61,63,68,69,72,79,82,89,92,94], interviews in 16 studies [13,27,30,32,35,37,44,48,50,57,65,68,73,76,77,93] and surveys or questionnaires in 10 studies [28,29,32,51,52,61,65,78,91,93]. Other techniques used included social media [45,68], observations in combination with a workshop or focus group, [13,63] an engagement event [26], a school visit [54], yarning circle or storytelling [62,70,94], and virtual simulation [50].

3.4. Those Involved in the Co-Production of Chronic Disease Prevention Interventions

Co-production was used to different extents depending on whether the intervention was in a development or implementation phase or whether the study was reporting on the evaluation of the intervention. We found 59 studies that reported using co-production in the development of an intervention, in a description akin to formative research [13,25,26,27,28,29,30,31,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,53,54,55,57,58,59,60,61,62,63,64,65,66,67,68,69,70,72,73,74,75,76,78,79,80,81,82,83,91,92,93,94]. In this development phase, we found academics (n = 55) [13,25,26,27,28,29,30,31,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,53,54,55,57,58,59,60,61,62,63,64,65,66,68,73,74,75,76,78,79,80,81,82,83,91,92,93,94], representatives from the target population (n = 54) [13,25,27,28,29,30,31,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,53,54,55,57,58,59,60,61,62,63,66,67,68,69,70,72,73,74,76,78,80,81,82,83,91,92,93,94], intervention designers/advisers (n = 23) [13,25,28,29,34,37,38,44,46,49,50,60,61,62,65,68,70,73,74,76,83,92,93], implementers of interventions (n = 22) [13,25,26,28,29,33,35,38,40,41,55,60,64,65,66,67,70,75,76,78,79,80] and policy makers (n = 6) [37,40,41,42,43,76].

Seven studies reported the use of co-production practices throughout the implementation phase of the intervention [52,54,56,71,76,80,83] involving academics (n = 4) [54,76,80,83], representatives from the target population (n = 3) [54,76,83], intervention designers/advisers (n = 2) [76,83], implementers (n = 2) [76,80] and policy makers [76]. The design of the intervention evaluation was reported in 14 studies [28,29,42,43,47,57,60,66,67,68,75,76,80,83] with academics as contributors in all of these studies. The evaluation was designed and conducted by academics alone in just two [66,68] out of these 14 studies despite multiple stakeholders being involved in the development of the intervention itself. When the evaluation was co-produced, representatives of the target population (n = 9) [28,29,42,43,47,57,75,76,83], intervention designers/advisers (n = 4) [28,29,76,83], implementers (n = 7) [28,29,60,67,75,76,80] and policy makers (n = 3) [42,43,76] worked alongside academics.

3.5. Evaluation of Chronic Disease Prevention Interventions Developed Using Co-Production

Ten studies included mention of the acceptability [26,31,51,60,92] and feasibility [26,27,31,36,47,51,58,60,73] of implementing a co-produced intervention. Among these studies, there was no consistency in the way acceptability and feasibility were measured. Most authors concluded that a co-produced intervention was feasible because the views of the target audience were able to be incorporated into a revised intervention. Similarly, acceptability was determined by implication because of the acceptability of the target audience’s involvement or as measured post-intervention development through questionnaires or qualitative interviews. Three protocol papers included plans to undertake a process evaluation, with some consideration given to issues of implementation of a co-produced intervention [32,35,77].

Eighteen studies reported on the evaluation of their intervention in terms of process evaluation (n = 5) [43,44,45,50,92], impact evaluation (n = 9) [47,57,65,66,67,69,71,78,79] and both process and impact evaluation (n = 4) [29,52,56,76]. These studies did not include an analysis of whether the use of co-production afforded any implementation or outcome advantage and, as such, did not report on the impact that co-production had on the associated implementation or outcomes of the program. A further two-thirds of the papers (65.3%, n = 47) included in the review limited their scope to describing only the techniques used for undertaking co-production, with no evaluation results reported of any kind.

4. Discussion

This scoping review found 71 articles that reported using co-production when developing a chronic disease prevention intervention or program, with the majority published in the last three years. Our findings highlight that different ‘co’- words were used interchangeably within and across many studies, and little attention was paid to whether there were any differences (subtle or otherwise) in their intended use and meaning. The ‘co’- words used included co-produce, co-design, co-create, co-develop, and co-construct, either singly or in combination. Although we initially focused on co-production, co-design and co-creation were more commonly used in practice in the selected primary studies. Occasionally, a ‘co’- word was used only in the abstract, perhaps as a way of drawing attention to the article, but the body text included no further exploration of the term.

Across the different terms, in the studies we reviewed, ‘co-‘ words were used to describe a process of engaging with the target audience of end users or intermediaries (e.g., health promotion and health practitioners, etc.) [95] of an intervention in a participatory fashion [17]. There were no substantive differences in meaning between co-design, co-create, co-produce, and co-construct and how they were deployed in reporting on an intervention. There were also no notable differences in the methods used in co-production based on the ‘co-‘ word used by the study. Overall, co-production constituted a formative research process [96], including focus groups, interviews, and other methods of information collection. Again, the literature would benefit from clarity as to whether the different terms are or should be linked in some way to particular techniques.

Through our analysis, it became apparent that those who use a co-production method choose the relevant stakeholders to be involved in the design, implementation, or evaluation of a chronic disease prevention intervention. The most common co-production participants were academics and the target audience, followed by intervention designers, implementers, and policymakers. This finding is not unexpected given that the majority of studies in our review reported on co-production in terms of the development of an intervention rather than the implementation and evaluation. It is uncertain whether there would be a benefit in attempting to define the group(s) to be involved in co-production processes, as this may vary widely with the project and context. More important is to examine the impact of including different groups on the outcomes and implementation of an intervention.

As noted above and by other authors [18,96], our review confirmed the paucity of evidence that examines the impact or effectiveness of co-production processes in chronic disease prevention interventions. This is unsurprising given that most studies included in this review outline the co-production technique, and the few that reported on evaluations used a pre-post design in relation to the interventions’ target behaviours. A few studies noted that using a co-production method was acceptable and feasible because a) the studies had been effective at incorporating the views of the target audience in the design of the intervention and b) post design, the intervention had been used by the target audience almost as a proxy measure of acceptability. Future studies should formally evaluate the perceived acceptability and feasibility of co-produced interventions within target populations rather than relying on proxy measures. The review papers we included also reported that studies were more likely to report on feasibility and acceptability rather than impact of the co-production process on intervention outcomes, with Eyles et al. [87] concluding that “sufficiency of reporting was poor, and no study undertook a robust assessment of efficacy” (p 160). Future studies with robust evaluation designs are needed to evaluate the effectiveness of co-produced health promotion interventions so the impact of the co-production method can be determined.

Our findings suggest that there would be merit in developing conceptual or definitional guidance as to what these words mean or include in the chronic disease prevention setting and whether there are differences in meaning or whether they can be used interchangeably to describe the same process. Our review supports the notion that ‘co’ is suggestive of a co-operative, collaborative, or participatory design [97], as noted by Blomkamp [98], but it is not possible to suggest from our findings any definite differences in meaning between the various ‘co’-words. There may be merit in developing a framework that provides greater understanding of the distinctions between various terminology. This could be progressed by borrowing from the health services [99,100] and public administration co-production literature [14,15] which provides some guidance in defining the most used co-words by articulating their differences and then providing a hierarchy of meanings that can be used to guide co-production in chronic disease prevention. Our research also suggests that starting points could be: defining terms by those involved in the collaborative process [6] or using a staged approach to co-production as mapped against a program design cycle [6,13]. There is also an opportunity to explore how co-production does or does not align with community-based participatory research [101] or participatory action research [102], particularly in relation to where along the intervention design and evaluation continuum it fits and also which stakeholders it involves.

This review provides an initial step in overviewing a growing field of research that is ‘messy’. While research co-design in health has been included in previous reviews [103], our review is novel in its focus on the use of co-production in the development and evaluation of co-produced chronic disease prevention interventions that aim to change a lifestyle behaviour. A strength is the inclusion of all ‘co-‘ words used in publications and the broad view taken to recognise similarities or differences in their use.

A number of limitations that may affect how the findings are interpreted need to be acknowledged. Only studies published in English were included, potentially excluding relevant studies published in other languages. The search was limited to peer-reviewed literature and did not include a grey literature search. It is possible that policy statements and reports relevant to co-produced chronic disease prevention interventions could have contributed to the review findings. Future reviews should include studies in other languages as well as from the grey literature. The wide range of study designs and research methods are drawn from for this review limited the options available for drawing conclusions for defining ‘co’-words and suggesting frameworks appropriate to health promotion interventions [21]. We did not conduct a quality appraisal of the primary studies included, which is consistent with scoping review methodology [19,21] but leads to a broad range of included material. The advantage of considering the breadth of literature is that it provides a structured overview of the current use of co-production in health promotion and provides direction for the focus of future research.

5. Implications for Practice and Research

Our review suggests that, as with co-production more broadly, co-produced health promotion interventions that aim to prevent chronic disease are not well described or robustly evaluated. The public health literature does not currently provide insight into whether co-produced interventions achieve better outcomes than those that are not co-produced [104]. Co-production, co-design, co-creation, and co-develop seem to be used interchangeably to refer to a participatory or collaborative process involving researchers, stakeholders, and end users involved in the development or evaluation of such interventions. Uniform agreement on the meanings of these words would avoid confusion surrounding their use and facilitate the development of guidelines and/or a co-production framework specific to health promotion interventions. Doing so would allow researchers to develop a shared understanding of the co-production process and how best to evaluate co-produced interventions.

Author Contributions

Conceptualisation, B.M., B.J.O., and L.C.; methodology, B.M., B.J.O., and L.C.; investigation and analysis, B.M., B.J.O., and L.C.; interpretation, B.M., B.J.O., L.C., A.C.G., and M.I.; writing—original draft preparation, B.M., B.J.O., and L.C.; writing—review and editing, B.M., B.J.O., L.C., A.C.G., and M.I. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors thank Philayrath Phongsavan for support with the initial conceptualisation of the study.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figure and Tables

Figure 1. Flowchart of study selection and exclusion.

Table 1

Processes used to facilitate input from stakeholders during co-production.

First Author, Year	Purpose of Study	Type of Co-Production	Prevention Focus	Target Population	Collaboration Technique
Carins 2021	Formative	Co-design, -create	Healthy eating	Supermarket consumers	Workshops
Hardt 2021	Formative	Co-design	Obesity prevention	Children	Survey, discussion groups, interviews
Mooses 2021	Formative	Co-create, -design	Physical activity	Children/adolescents (7–16 years)	Network building, school visits
Ochieng 2021	Formative	Co-create	Healthy weight	Children (ethnic minority)	Focus groups, workshops
Castro 2020	Formative	Co-design, -create	Physical activity	Low-income adults (40–90 years)	Focus groups
Champion 2020	Formative	Co-design	Lifestyle risk factors	Secondary school students	Survey, focus groups
Corr 2020	Formative, impact	Co-create	Physical activity	Adolescent girls (15–17 years)	Questionnaire, focus groups
D’Addario 2020	Formative	Co-design	Physical activity	Physically inactive adults	Focus groups
Daly-Smith 2020	Formative	Co-produce, -design, -develop	Physical activity	School-aged children/adolescents	Stakeholder workshops
Hidding 2020	Formative	Co-create	Physical activity	Children (9–12 years old)	Concept mapping, focus groups
Martin 2020	Formative	Co-design	Healthy weight	Adolescents (13–16 years)	Workshop, individual testing
Parder 2020	Formative	Co-create, -produce	Alcohol abuse prevention	Adolescents (13–15 years)	Workshops, storytelling
Peiris-Hohn 2020	Formative, process	Co-design, co-create	‘Health’ including PA	Adolescents (16+ years)	Group sessions, workshops
Lems 2020Lems 2019	FormativeFormative	Co-create	Health promotion	Adolescent girls (12–15 years) and boys (12–18 years)	Small group sessions
Anselma 2019Anselma 2020	FormativeProcess	Co-design, -createCo-create, -develop	Healthy lifestyle	Children (9–12 years)	Group sessions
Fournier 2019	Formative, process	Co-construct	Physical inactivity	Older adults	Group sessions, interviews
Gillespie 2019	Formative	Co-produce	Obesity prevention	Primary school-aged children	Focus groups, interviews
Goffe 2019	Formative	Co-design	Food portion sizing	Food outlet owners/managers	Discussions, engagement event
Gould 2019	Formative	Co-design	Smoking cessation	Pregnant Indigenous women	Not specified
Hoeeg 2019	Formative	Co-design, -create	Obesity prevention	Families of preschool children	Workshops
Mammen 2019	Formative	Co-create	Health messages	Rural, low-income mothers	Focus groups, interviews
Mistura 2019	Formative, impact	Co-design	Food purchasing	First-year university students	Focus groups, surveys
Morgan 2019	Formative	Co-produce	Physical activity	Girls (9–11 years)	Focus groups, interviews
Ojo 2019	Formative	Co-create	Workplace sitting	Desk-based workers	Interviews
Partridge 2019	Formative	Co-design	Obesity prevention	Adolescents (13–18 years)	Workshop, survey
Santina 2019	Formative	Co-design, -develop	Physical activity	Children (10–12 years)	Group meetings
Buckley 2018Buckley 2019	FormativeProcess, impact	Co-develop, -produceCo-produce	Physical activity	Adults with controlled lifestyle-related health issues	Group meetings, focus groups, survey
Guell 2018	Formative	Co-design, -develop	Physical activity	Older adults	Interviews, workshops
Street 2018	Formative	Co-construct, -create, -produce	Health policy	Aboriginal people	Deliberative forum, storyboard
Te Morenga 2018	Formative	Co-design	Obesity prevention	Maori people	Focus groups
Verbiest 2018	Protocol	Co-design	Healthy lifestyle behaviour	Adult Maori people
Durl 2017	Formative	Co-design	Alcohol education	Adolescents (14–16 years)	Workshop, feedback, observations
Hawkins 2017	Formative	Co-produce	Smoking prevention	Adolescents (12–19 years)	Focus groups, interviews, observations
Janols 2017	Formative	Co-design	Health behaviour change	Older adults	Workshops
Leask 2017	Formative	Co-create	Sedentary behaviour	Older adults	Workshops
Verloigne 2017	Formative, impact	Co-create	Physical activity	Adolescent girls (16 years)	Groups
Yuan 2017	Formative	Co-create	Physical activity	Older adults	Workshops
Chau 2016	Formative	Co-design	Sedentary behaviour	Adult call-centre workers	Not specified
Nu 2016	Formative	Co-design	Dietary pattern change	Indigenous community	Working group
Rosso 2016	Formative, impact	Co-design	Health promotion (sport)	Children and youth	Interviews, surveys
Standoli 2016	Formative	Co-design	Obesity prevention	Adolescents	Focus groups
Isbell 2015	Formative	Co-create	Nutrition education	Women, infants, children	Strategic planning meetings
Mackenzie 2015	Formative	Co-produce	Sitting	University employees	Not specified
Vallentin-Holbech 2020	Process	Co-create	Alcohol consumption	Adolescents (15–18 years)	Workshops, interviews, virtual simulation
van den Heerik 2017	Process	Co-create	Smoking prevention	Youth (15–25 years)	Social media, linguistic analysis
Ahmed 2020	Impact, process	Co-design	Healthy eating	Indigenous tribal community	Focus group interviews
Bogomolova 2021	Impact	Co-design, -create	Healthy eating	Supermarket consumers	Workshops
Brimblecombe 2020	Impact	Co-design	Healthy eating	Remote Aboriginal communities	Working groups
Gallegos 2020	Impact	Co-design	Chronic disease	Ethnic communities	Not specified
De Rosis 2020	Impact, process	Co-produce, -design	Obesity prevention	Adolescents	Questionnaire (for evaluation)
Skerletopoulos 2020	Impact, process	Co-create	Smoking indoors	Citizens, commercial stakeholders
Fehring 2019	Impact	Co-design	Water consumption	Remote Aboriginal communities	Group meetings
McKay 2018	Impact	Co-design	Physical activity	Older adults	Workshops
Perignon 2017	Impact, formative	Co-construct	Healthy eating	Socioeconomic disadvantage	Workshops, interviews
Beckerman-Hsu 2020	Protocol (process)	Co-design	Obesity prevention	Low-income preschool children	Focus groups, interviews
Bovill 2021	Protocol (formative)	Co-design, -develop	Smoking cessation	Pregnant Aboriginal women	Yarning circles, e-mail survey
Latomme 2021	Protocol (formative)	Co-create	Physical activity	Fathers and their children	Group sessions, interviews
Nahar 2020	Protocol (process)	Co-produce, -design	Cardiovascular prevention	Disadvantaged populations	Focus groups, questionnaires, interviews
Folkvord 2019	Protocol (formative)	Co-create	Fruit and vegetable intake	Children (7–13 years)	Focus groups
Lombard 2018	Protocol (formative)	Co-design, -create	Healthy eating	Young adults (18–24 years)	Social media, interviews, workshops
Gillespie 2019	Protocol (process)	Co-produce	Obesity prevention	Preschool-aged children	Group meetings
Review style papers
Taggart 2021	Review	Co-produce	Obesity	Adults (intellectual disabilities)	Workshops
Ruan 2020	Review	Co-design	Health behaviours	Adolescents	Content analysis
Rutten 2019	Review, comment	Co-produce	Active lifestyles	Population-wide	Systems approach
Partridge 2018	Review	Co-design	Healthy lifestyle	Adolescents	Focus groups, interviews
Raeside 2018	Review	Co-create	Healthy behaviours	Adolescents	Focus groups, workshops
Taggart 2018	Review	Co-design, -develop, -produce	Type 2 diabetes prevention	Adults (intellectual disabilities)	Focus groups, interviews
Eyles 2016	Review	Co-design	Health behaviour change	Not limited by population	Not limited by collaborative technique

Note: Studies are organised by purpose of study, and within the broad purpose, they are organised alphabetically by year with the most recent first.

Table 2

Techniques used in co-produced interventions.

Technique	Co-Design	Co-Create	Co-Produce	Co-Construct	Combination	Total
	n (%)	n (%)	n (%)	n (%)	n (%)	N
Group session	12 (34)	10 (28.6)	4 (11.4)	1 (2.9)	8 (22.9)	35
Workshop	7 (35.0)	5 (25.0)	2 (10.0)		6 (30.0)	20
Interviews	4 (25.0)	4 (25.0)	3 (18.8)	2 (12.5)	3 (18.8)	16
Survey/questionnaire	5 (45.5)	2 (18.2)			4 (36.4)	11
Storytelling					3 (100.0)	3
Social media		1 (50.0)			1 (50.0)	2
Observation	1 (50.0)		1 (50.0)			2
Event	1 (100.0)					1
School visit					1 (100.0)	1
Virtual simulation		1 (100.0)				1

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/healthcare10040647/s1, Table S1: Search strategy; Table S2: Data extraction table.

References

1. Australian Institute of Health and Welfare. Impact of Overweight and Obesity as a Risk Factor for Chronic Conditions: Australian Burden of Disease Study; AIHW: Canberra, ACT, Australia, 2017.

2. Australian Institute of Health and Welfare. Australian Burden of Disease Study: Impact and Causes of Illness and Death in Australia 2015; AIHW: Canberra, ACT, Australia, 2019.

3. World Health Organization. Noncommunicable Diseases Progress Monitor 2020; World Health Organization: Geneva, Switzerland, 2020.

4. Australian Government Department of Health. Preventive Health. 2020; Available online: https://www.health.gov.au/health-topics/preventive-health (accessed on 11 January 2021).

5. Wutzke, S.; Rowbotham, S.; Haynes, A.; Hawe, P.; Kelly, P.; Redman, S.; Davidson, S.; Stephenson, J.; Overs, M.; Wilson, A. Knowledge mobilisation for chronic disease prevention: The case of the Australian Prevention Partnership Centre. Health Res. Policy Syst.; 2018; 16, pp. 1-16. [DOI: https://dx.doi.org/10.1186/s12961-018-0379-9]

6. Elwyn, G.; Nelson, E.; Hager, A.; Price, A. Coproduction: When users define quality. BMJ Qual. Saf.; 2020; 29, pp. 711-716. [DOI: https://dx.doi.org/10.1136/bmjqs-2019-009830] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31488570]

7. Haynes, A.; Rowbotham, S.; Grunseit, A.; Bohn-Goldbaum, E.; Slaytor, E.; Wilson, A.; Lee, K.; Davidson, S.; Wutzke, S. Knowledge mobilisation in practice: An evaluation of the Australian Prevention Partnership Centre. Health Res. Policy Syst.; 2020; 18, pp. 1-17. [DOI: https://dx.doi.org/10.1186/s12961-019-0496-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32005254]

8. Turakhia, P.; Combs, B. Using Principles of Co-Production to Improve Patient Care and Enhance Value. AMA J. Ethic; 2017; 19, pp. 1125-1131. [DOI: https://dx.doi.org/10.1001/journalofethics.2017.19.11.pfor1-1711]

9. Fusco, F.; Marsilio, M.; Guglielmetti, C. Co-production in health policy and management: A comprehensive bibliometric review. BMC Health Serv. Res.; 2020; 20, pp. 1-16. [DOI: https://dx.doi.org/10.1186/s12913-020-05241-2] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32503522]

10. Clarke, D.; Jones, F.; Harris, R.; Robert, G. What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings?. A rapid evidence synthesis. BMJ Open; 2017; 7, e014650. [DOI: https://dx.doi.org/10.1136/bmjopen-2016-014650]

11. DuRose, C.; Needham, C.; Mangan, C.; Rees, J. Generating ’good enough’ evidence for co-production. Évid. Policy A J. Res. Debate Pract.; 2017; 13, pp. 135-151. [DOI: https://dx.doi.org/10.1332/174426415X14440619792955]

12. Beckett, K.; Farr, M.; Kothari, A.; Wye, L.; Le May, A. Embracing complexity and uncertainty to create impact: Exploring the processes and transformative potential of co-produced research through development of a social impact model. Health Res. Policy Syst.; 2018; 16, pp. 1-18. [DOI: https://dx.doi.org/10.1186/s12961-018-0375-0]

13. Hawkins, J.; Madden, K.; Fletcher, A.; Midgley, L.; Grant, A.; Cox, G.; Moore, L.; Campbell, R.; Murphy, S.; Bonell, C. et al. Development of a framework for the co-production and prototyping of public health interventions. BMC Public Health; 2017; 17, 689. [DOI: https://dx.doi.org/10.1186/s12889-017-4695-8]

14. Osborne, S.P.; Radnor, Z.; Strokosch, K. Co-Production and the Co-Creation of Value in Public Services: A suitable case for treatment?. Public Manag. Rev.; 2016; 18, pp. 639-653. [DOI: https://dx.doi.org/10.1080/14719037.2015.1111927]

15. Jo, S.; Nabatchi, T. Getting Back to Basics: Advancing the Study and Practice of Coproduction. Int. J. Public Adm.; 2016; 39, pp. 1-8. [DOI: https://dx.doi.org/10.1080/01900692.2016.1177840]

16. Nabatchi, T.; Sancino, A.; Sicilia, M. Varieties of Participation in Public Services: The Who, When, and What of Coproduction. Public Adm. Rev.; 2017; 77, pp. 766-776. [DOI: https://dx.doi.org/10.1111/puar.12765]

17. Green, L.W.; O’Neill, M.; Westphal, M.; Morisky, D.; Editors, G. The Challenges of Participatory Action Research for Health Promotion; Sage Publications: Thousand Oaks, CA, USA, 1996.

18. Redman, S.; Greenhalgh, T.; Adedokun, L.; Staniszewska, S.; Denegri, S. Co-production of knowledge: The future. BMJ; 2021; 372, n434. [DOI: https://dx.doi.org/10.1136/bmj.n434]

19. Grant, M.J.; Booth, A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Inf. Libr. J.; 2009; 26, pp. 91-108. [DOI: https://dx.doi.org/10.1111/j.1471-1842.2009.00848.x]

20. Peters, M.D.J.; Godfrey, C.M.; Khalil, H.; McInerney, P.; Parker, D.; Soares, C.B. Guidance for conducting systematic scoping reviews. Int. J. Evid. Based Healthc.; 2015; 13, pp. 141-146. [DOI: https://dx.doi.org/10.1097/XEB.0000000000000050]

21. Arksey, H.; O’Malley, L. Scoping studies: Towards a methodological framework. Int. J. Soc. Res. Methodol.; 2005; 8, pp. 19-32. [DOI: https://dx.doi.org/10.1080/1364557032000119616]

22. Levac, D.; Colquhoun, H.; O’Brien, K.K. Scoping studies: Advancing the methodology. Implement. Sci.; 2010; 5, pp. 1-9. [DOI: https://dx.doi.org/10.1186/1748-5908-5-69]

23. Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L. et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann. Intern. Med.; 2018; 169, pp. 467-473. [DOI: https://dx.doi.org/10.7326/M18-0850]

24. Johnston, L.M.; Finegood, D.T. Cross-Sector Partnerships and Public Health: Challenges and Opportunities for Addressing Obesity and Noncommunicable Diseases Through Engagement with the Private Sector. Annu. Rev. Public Health; 2015; 36, pp. 255-271. [DOI: https://dx.doi.org/10.1146/annurev-publhealth-031914-122802]

25. Daly-Smith, A.; Quarmby, T.; Archbold, V.S.J.; Corrigan, N.; Wilson, D.; Resaland, G.K.; Bartholomew, J.B.; Singh, A.; Tjomsland, H.E.; Sherar, L.B. et al. Using a multi-stakeholder experience-based design process to co-develop the Creating Active Schools Framework. Int. J. Behav. Nutr. Phys. Act.; 2020; 17, 13. [DOI: https://dx.doi.org/10.1186/s12966-020-0917-z]

26. Goffe, L.; Hillier-Brown, F.; Hildred, N.; Worsnop, M.; Adams, J.; Araujo-Soares, V.; Penn, L.; Wrieden, W.; Summerbell, C.; A Lake, A. et al. Feasibility of working with a wholesale supplier to co-design and test acceptability of an intervention to promote smaller portions: An uncontrolled before-and-after study in British Fish & Chip shops. BMJ Open; 2019; 9, e023441. [DOI: https://dx.doi.org/10.1136/bmjopen-2018-023441] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30782880]

27. Ojo, S.O.; Bailey, D.P.; Brierley, M.L.; Hewson, D.J.; Chater, A.M. Breaking barriers: Using the behavior change wheel to develop a tailored intervention to overcome workplace inhibitors to breaking up sitting time. BMC Public Health; 2019; 19, pp. 1-17. [DOI: https://dx.doi.org/10.1186/s12889-019-7468-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31420033]

28. Buckley, B.J.R.; Thijssen, D.H.J.; Murphy, R.C.; Graves, L.; Whyte, G.; Gillison, F.B.; Crone, D.; Wilson, P.M.; Watson, P.M. Making a move in exercise referral: Co-development of a physical activity referral scheme. J. Public Health; 2018; 40, pp. e586-e593. [DOI: https://dx.doi.org/10.1093/pubmed/fdy072] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29688551]

29. Buckley, B.J.; Thijssen, D.H.; Murphy, R.C.; Graves, L.; Whyte, G.; Gillison, F.; Crone, D.; Wilson, P.M.; Hindley, D.; Watson, P.M. Preliminary effects and acceptability of a co-produced physical activity referral intervention. Health Educ. J.; 2019; 78, pp. 869-884. [DOI: https://dx.doi.org/10.1177/0017896919853322]

30. Guell, C.; Panter, J.; Griffin, S.; Ogilvie, D. Towards co-designing active ageing strategies: A qualitative study to develop a meaningful physical activity typology for later life. Health Expect.; 2018; 21, pp. 919-926. [DOI: https://dx.doi.org/10.1111/hex.12686]

31. MacKenzie, K.; Goyder, E.; Eves, F. Acceptability and feasibility of a low-cost, theory-based and co-produced intervention to reduce workplace sitting time in desk-based university employees. BMC Public Health; 2015; 15, pp. 1-13. [DOI: https://dx.doi.org/10.1186/s12889-015-2635-z]

32. Nahar, P.; van Marwijk, H.; Gibson, L.; Musinguzi, G.; Anthierens, S.; Ford, E.; Bremner, S.A.; Bowyer, M.; Le Reste, J.Y.; Sodi, T. et al. A protocol paper: Community engagement interventions for cardiovascular disease prevention in socially disadvantaged populations in the UK: An implementation research study. Glob. Health Res. Policy; 2020; 5, pp. 1-9. [DOI: https://dx.doi.org/10.1186/s41256-020-0131-1]

33. Ochieng, L.; Amaugo, L.; Ochieng, B.M.N. Developing healthy weight maintenance through co-creation: A partnership with Black African migrant community in East Midlands. Eur. J. Public Health; 2021; 31, pp. 487-493. [DOI: https://dx.doi.org/10.1093/eurpub/ckaa222]

34. Gillespie, J.; Hughes, A.; Gibson, A.-M.; Haines, J.; Taveras, E.; Reilly, J.J. Protocol for Healthy Habits Happy Homes (4H) Scotland: Feasibility of a participatory approach to adaptation and implementation of a study aimed at early prevention of obesity. BMJ Open; 2019; 9, e028038. [DOI: https://dx.doi.org/10.1136/bmjopen-2018-028038]

35. Gillespie, J.; Magee, E.; White, A.; Stewart, L. Eat, play, learn well—a novel approach to co-production and analysis grid for environments linked to obesity to engage local communities in a child healthy weight action plan. Public Health; 2019; 166, pp. 99-107. [DOI: https://dx.doi.org/10.1016/j.puhe.2018.09.032]

36. Leask, C.F.; Sandlund, M.; A Skelton, D.; Chastin, S.F. Co-creating a tailored public health intervention to reduce older adults’ sedentary behaviour. Health Educ. J.; 2017; 76, pp. 595-608. [DOI: https://dx.doi.org/10.1177/0017896917707785]

37. Morgan, K.; Van Godwin, J.; Darwent, K.; Fildes, A. Formative research to develop a school-based, community-linked physical activity role model programme for girls: CHoosing Active Role Models to INspire Girls (CHARMING). BMC Public Health; 2019; 19, pp. 1-13. [DOI: https://dx.doi.org/10.1186/s12889-019-6741-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31023293]

38. Lu, Y.; Valk, C.; Steenbakkers, J.; Bekker, T.; Visser, T.; Proctor, G.; Toshniwal, O.; Langberg, H. Can technology adoption for older adults be co-created?. Gerontechnology; 2017; 16, pp. 151-159. [DOI: https://dx.doi.org/10.4017/gt.2017.16.3.004.00]

39. Hidding, L.M.; Chinapaw, M.J.M.; Belmon, L.S.; Altenburg, T.M. Co-creating a 24-hour movement behavior tool together with 9–12-year-old children using mixed-methods: MyDailyMoves. Int. J. Behav. Nutr. Phys. Act.; 2020; 17, pp. 1-12. [DOI: https://dx.doi.org/10.1186/s12966-020-00965-0]

40. Lems, E.; Hilverda, F.; Broerse, J.E.W.; Dedding, C. ‘Just stuff yourself’: Identifying health-promotion strategies from the perspectives of adolescent boys from disadvantaged neighbourhoods. Health Expect.; 2019; 22, pp. 1040-1049. [DOI: https://dx.doi.org/10.1111/hex.12913]

41. Lems, E.; Hilverda, F.; Sarti, A.; Van Der Voort, L.; Kegel, A.; Pittens, C.; Broerse, J.; Dedding, C. ‘McDonald’s Is Good for My Social Life’. Developing Health Promotion Together with Adolescent Girls from Disadvantaged Neighbourhoods in Amsterdam. Child. Soc.; 2020; 34, pp. 204-219. [DOI: https://dx.doi.org/10.1111/chso.12368]

42. Anselma, M.; Altenburg, T.M.; Emke, H.; Van Nassau, F.; Jurg, M.; Ruiter, R.A.C.; Jurkowski, J.M.; Chinapaw, M.J.M. Co-designing obesity prevention interventions together with children: Intervention mapping meets youth-led participatory action research. Int. J. Behav. Nutr. Phys. Act.; 2019; 16, pp. 1-15. [DOI: https://dx.doi.org/10.1186/s12966-019-0891-5]

43. Anselma, M.; Chinapaw, M.; Altenburg, T. “Not Only Adults Can Make Good Decisions, We as Children Can Do That as Well” Evaluating the Process of the Youth-Led Participatory Action Research ‘Kids in Action’. Int. J. Environ. Res. Public Health; 2020; 17, 625. [DOI: https://dx.doi.org/10.3390/ijerph17020625]

44. Fournier, B.; Manon, P.; Johanne, F.; Nathalie, B.; Lorthios-Guilledroit, A.; Marie-Ève, M. Development and implementation of a community-based pole walking program for older adults. Act. Adapt. Aging; 2019; 43, pp. 1-22. [DOI: https://dx.doi.org/10.1080/01924788.2018.1428471]

45. van den Heerik, R.A.M.; van Hooijdonk, C.M.J.; Burgers, C.; Steen, G.J. “Smoking Is Sooo. Sandals and White Socks”: Co-Creation of a Dutch Anti-Smoking Campaign to Change Social Norms. Health Commun.; 2017; 32, pp. 621-628. [DOI: https://dx.doi.org/10.1080/10410236.2016.1168000]

46. Folkvord, F. Systematically testing the effects of promotion techniques on children’s fruit and vegetables intake on the long term: A protocol study of a multicenter randomized controlled trial. BMC Public Health; 2019; 19, pp. 1-7. [DOI: https://dx.doi.org/10.1186/s12889-019-7952-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31775699]

47. Verloigne, M.; Altenburg, T.M.; Chinapaw, M.J.M.; Chastin, S.; Cardon, G.; De Bourdeaudhuij, I. Using a Co-Creational Approach to Develop, Implement and Evaluate an Intervention to Promote Physical Activity in Adolescent Girls from Vocational and Technical Schools: A Case Control Study. Int. J. Environ. Res. Public Health; 2017; 14, 862. [DOI: https://dx.doi.org/10.3390/ijerph14080862] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28763041]

48. Latomme, J.; Morgan, P.; De Craemer, M.; Brondeel, R.; Verloigne, M.; Cardon, G. A Family-Based Lifestyle Intervention Focusing on Fathers and Their Children Using Co-Creation: Study Protocol of the Run Daddy Run Intervention. Int. J. Environ. Res. Public Health; 2021; 18, 1830. [DOI: https://dx.doi.org/10.3390/ijerph18041830] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33668562]

49. Hoeeg, D.; Christensen, U.; Grabowski, D. Co-Designing an Intervention to Prevent Overweight and Obesity among Young Children and Their Families in a Disadvantaged Municipality: Methodological Barriers and Potentials. Int. J. Environ. Res. Public Health; 2019; 16, 5110. [DOI: https://dx.doi.org/10.3390/ijerph16245110]

50. Vallentin-Holbech, L.; Guldager, J.D.; Dietrich, T.; Rundle-Thiele, S.; Majgaard, G.; Lyk, P.; Stock, C. Co-Creating a Virtual Alcohol Prevention Simulation with Young People. Int. J. Environ. Res. Public Health; 2020; 17, 1097. [DOI: https://dx.doi.org/10.3390/ijerph17031097]

51. Corr, M.; Murtagh, E. ‘No one ever asked us’: A feasibility study assessing the co-creation of a physical activity programme with adolescent girls. Glob. Health Promot.; 2020; 27, pp. 34-43. [DOI: https://dx.doi.org/10.1177/1757975919853784]

52. De Rosis, S.; Pennucci, F.; Noto, G.; Nuti, S. Healthy Living and Co-Production: Evaluation of Processes and Outcomes of a Health Promotion Initiative Co-Produced with Adolescents. Int. J. Environ. Res. Public Health; 2020; 17, 8007. [DOI: https://dx.doi.org/10.3390/ijerph17218007]

53. D’Addario, M.; Baretta, D.; Zanatta, F.; Greco, A.; Steca, P. Engagement Features in Physical Activity Smartphone Apps: Focus Group Study with Sedentary People. JMIR mHealth uHealth; 2020; 8, e20460. [DOI: https://dx.doi.org/10.2196/20460]

54. Mooses, K.; Vihalemm, T.; Uibu, M.; Mägi, K.; Korp, L.; Kalma, M.; Mäestu, E.; Kull, M. Developing a comprehensive school-based physical activity program with flexible design—from pilot to national program. BMC Public Health; 2021; 21, pp. 1-14. [DOI: https://dx.doi.org/10.1186/s12889-020-10111-x]

55. Janols, R.; Lindgren, H. A Method for Co-Designing Theory-Based Behaviour Change Systems for Health Promotion. Stud. Health Technol. Inform.; 2017; 235, pp. 368-372.

56. Skerletopoulos, L.; Makris, A.; Khaliq, M. “Trikala Quits Smoking”: A Citizen Co-Creation Program Design to Enforce the Ban on Smoking in Enclosed Public Spaces in Greece. Soc. Mark. Q.; 2020; 26, pp. 189-203. [DOI: https://dx.doi.org/10.1177/1524500420942437]

57. Perignon, M.; Dubois, C.; Gazan, R.; Maillot, M.; Muller, L.; Ruffieux, B.; Gaigi, H.; Darmon, N. Co-construction and Evaluation of a Prevention Program for Improving the Nutritional Quality of Food Purchases at No Additional Cost in a Socioeconomically Disadvantaged Population. Curr. Dev. Nutr.; 2017; 1, e001107. [DOI: https://dx.doi.org/10.3945/cdn.117.001107] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29955680]

58. Martin, A.; Caon, M.; Adorni, F.; Andreoni, G.; Ascolese, A.; Atkinson, S.; Bul, K.; Carrion, C.; Castell, C.; Ciociola, V. et al. A Mobile Phone Intervention to Improve Obesity-Related Health Behaviors of Adolescents Across Europe: Iterative Co-Design and Feasibility Study. JMIR mHealth uHealth; 2020; 8, e14118. [DOI: https://dx.doi.org/10.2196/14118] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32130179]

59. Standoli, C.E.; Guarneri, M.R.; Perego, P.; Mazzola, M.; Mazzola, A.; Andreoni, G. Smart Wearable Sensor System for Counter-Fighting Overweight in Teenagers. Sensors; 2016; 16, 1220. [DOI: https://dx.doi.org/10.3390/s16081220] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27517929]

60. Gould, G.S.; Bovill, M.; Pollock, L.; Bonevski, B.; Gruppetta, M.; Atkins, L.; Carson-Chahhoud, K.; Boydell, K.M.; Gribbin, G.R.; Oldmeadow, C. et al. Feasibility and acceptability of Indigenous Counselling and Nicotine (ICAN) QUIT in Pregnancy multicomponent implementation intervention and study design for Australian Indigenous pregnant women: A pilot cluster randomised step-wedge trial. Addict. Behav.; 2019; 90, pp. 176-190. [DOI: https://dx.doi.org/10.1016/j.addbeh.2018.10.036] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30412909]

61. Partridge, S.R.; Raeside, R.; Latham, Z.; Singleton, A.C.; Hyun, K.; Grunseit, A.; Steineck, K.; Redfern, J. ’Not to Be Harsh but Try Less to Relate to ’the Teens’ and You’ll Relate to Them More’: Co-Designing Obesity Prevention Text Messages with Adolescents. Int. J. Environ. Res. Public Health; 2019; 16, 4887. [DOI: https://dx.doi.org/10.3390/ijerph16244887] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31817167]

62. Street, J.; Cox, H.; Lopes, E.; Motlik, J.; Hanson, L. Supporting youth wellbeing with a focus on eating well and being active: Views from an Aboriginal community deliberative forum. Aust. New Zealand J. Public Health; 2018; 42, pp. 127-132. [DOI: https://dx.doi.org/10.1111/1753-6405.12763]

63. Durl, J.; Trischler, J.; Dietrich, T. Co-designing with young consumers—Reflections, challenges and benefits. Young Consum.; 2017; 18, pp. 439-455. [DOI: https://dx.doi.org/10.1108/YC-08-2017-00725]

64. Chau, J.Y.; Engelen, L.; Burks-Young, S.; Daley, M.; Maxwell, J.-K.; Milton, K.; Bauman, A. Perspectives on a ‘Sit Less, Move More’ Intervention in Australian Emergency Call Centres. AIMS Public Health; 2016; 3, pp. 288-297. [DOI: https://dx.doi.org/10.3934/publichealth.2016.2.288]

65. Rosso, E.; McGrath, R. Promoting physical activity among children and youth in disadvantaged South Australian CALD communities through alternative community sport opportunities. Health Promot. J. Aust.; 2016; 27, pp. 105-110. [DOI: https://dx.doi.org/10.1071/HE15092]

66. Gallegos, D.; Do, H.; To, Q.G.; Vo, B.; Goris, J.; Alraman, H. The effectiveness of living well multicultural-lifestyle management program among ethnic populations in Queensland, Australia. Health Promot. J. Aust.; 2021; 32, pp. 84-95. [DOI: https://dx.doi.org/10.1002/hpja.329] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32053254]

67. Fehring, E.; Ferguson, M.; Brown, C.; Murtha, K.; Laws, C.; Cuthbert, K.; Thompson, K.; Williams, T.; Hammond, M.; Brimblecombe, J. Supporting healthy drink choices in remote Aboriginal and Torres Strait Islander communities: A community-led supportive environment approach. Aust. N. Z. J. Public Health; 2019; 43, pp. 551-557. [DOI: https://dx.doi.org/10.1111/1753-6405.12950] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31667933]

68. Lombard, C.; Brennan, L.; Reid, M.; Klassen, K.M.; Palermo, C.; Walker, T.; Lim, M.S.; Dean, M.; McCaffrey, T.A.; Truby, H. Communicating health-Optimising young adults’ engagement with health messages using social media: Study protocol. Nutr. Diet.; 2018; 75, pp. 509-519. [DOI: https://dx.doi.org/10.1111/1747-0080.12448] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30009396]

69. Bogomolova, S.; Carins, J.; Dietrich, T.; Bogomolov, T.; Dollman, J. Encouraging healthier choices in supermarkets: A co-design approach. Eur. J. Mark.; 2021; 55, pp. 2439-2463. [DOI: https://dx.doi.org/10.1108/EJM-02-2020-0143]

70. Bovill, M.; Chamberlain, C.; Bennett, J.; Longbottom, H.; Bacon, S.; Field, B.; Hussein, P.; Berwick, R.; Gould, G.; O’mara, P. Building an Indigenous-Led Evidence Base for Smoking Cessation Care among Aboriginal and Torres Strait Islander Women during Pregnancy and Beyond: Research Protocol for the Which Way?. Project. Int. J. Environ. Res. Public Health; 2021; 18, 1342. [DOI: https://dx.doi.org/10.3390/ijerph18031342] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33540747]

71. Brimblecombe, J.; McMahon, E.; Ferguson, M.; De Silva, K.; Peeters, A.; Miles, E.; Wycherley, T.; Minaker, L.; Greenacre, L.; Gunther, A. et al. Effect of restricted retail merchandising of discretionary food and beverages on population diet: A pragmatic randomised controlled trial. Lancet Planet. Health; 2020; 4, pp. e463-e473. [DOI: https://dx.doi.org/10.1016/S2542-5196(20)30202-3]

72. Carins, J.; Bogomolova, S. Co-designing a community-wide approach to encouraging healthier food choices. Appetite; 2021; 162, 105167. [DOI: https://dx.doi.org/10.1016/j.appet.2021.105167]

73. Mammen, S.; Sano, Y.; Braun, B.; Maring, E.F. Shaping Core Health Messages: Rural, Low-Income Mothers Speak through Participatory Action Research. Health Commun.; 2019; 34, pp. 1141-1149. [DOI: https://dx.doi.org/10.1080/10410236.2018.1465792]

74. Nu, J.; Bersamin, A. Collaborating with Alaska Native Communities to Design a Cultural Food Intervention to Address Nutrition Transition. Prog. Community Health Partnersh. Res. Educ. Action; 2017; 11, pp. 71-80. [DOI: https://dx.doi.org/10.1353/cpr.2017.0009]

75. Isbell, M.; Seth, J.G.; Atwood, R.D.; Ray, T.C. Development and Implementation of Client-Centered Nutrition Education Programs in a 4-Stage Framework. Am. J. Public Health; 2015; 105, pp. e65-e70. [DOI: https://dx.doi.org/10.2105/AJPH.2014.302512]

76. Ahmed, S.; Dupuis, V.; Tyron, M.; Crane, M.R.; Garvin, T.; Pierre, M.; Shanks, C.B. Intended and Unintended Consequences of a Community-Based Fresh Fruit and Vegetable Dietary Intervention on the Flathead Reservation of the Confederated Salish and Kootenai Tribes. Front. Public Health; 2020; 8, 331. [DOI: https://dx.doi.org/10.3389/fpubh.2020.00331] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32850578]

77. Beckerman-Hsu, J.P.; Aftosmes-Tobio, A.; Gavarkovs, A.; Kitos, N.; Figueroa, R.; Kalyoncu, Z.B.; Lansburg, K.; Yu, X.; Kazik, C.; Vigilante, A. et al. Communities for Healthy Living (CHL) A Community-based Intervention to Prevent Obesity in Low-Income Preschool Children: Process Evaluation Protocol. Trials; 2020; 21, pp. 1-13. [DOI: https://dx.doi.org/10.1186/s13063-020-04571-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32703293]

78. Mistura, M.; Fetterly, N.; Rhodes, R.E.; Tomlin, D.; Naylor, P.-J. Examining the Efficacy of a ’Feasible’ Nudge Intervention to Increase the Purchase of Vegetables by First Year University Students (17–19 Years of Age) in British Columbia: A Pilot Study. Nutrients; 2019; 11, 1786. [DOI: https://dx.doi.org/10.3390/nu11081786] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31382395]

79. McKay, H.; Nettlefold, L.; Bauman, A.; Hoy, C.; Gray, S.M.; Lau, E.; Sims-Gould, J. Implementation of a co-designed physical activity program for older adults: Positive impact when delivered at scale. BMC Public Health; 2018; 18, pp. 1-15. [DOI: https://dx.doi.org/10.1186/s12889-018-6210-2]

80. Te Morenga, L.; Pekepo, C.; Corrigan, C.; Matoe, L.; Mules, R.; Goodwin, D.; Dymus, J.; Tunks, M.; Grey, J.; Humphrey, G. et al. Co-designing an m, Health tool in the New Zealand Māori community with a “Kaupapa Māori” approach. AlterNative Int. J. Indig. Peoples; 2018; 14, pp. 90-99. [DOI: https://dx.doi.org/10.1177/1177180117753169]

81. Verbiest, M.; Borrell, S.; Dalhousie, S.; Tupa’I-Firestone, R.; Funaki, T.; Goodwin, D.; Grey, J.; Henry, A.; Hughes, E.; Humphrey, G. et al. A Co-Designed, Culturally-Tailored mHealth Tool to Support Healthy Lifestyles in Māori and Pasifika Communities in New Zealand: Protocol for a Cluster Randomized Controlled Trial. JMIR Res. Protoc.; 2018; 7, e10789. [DOI: https://dx.doi.org/10.2196/10789]

82. Castro, P.C.; Romano, L.B.; Frohlich, D.; Lorenzi, L.J.; Campos, L.B.; Paixão, A.; Bet, P.; Deutekom, M.; Krose, B.; Dourado, V.Z. et al. Tailoring digital apps to support active ageing in a low income community. PLoS ONE; 2020; 15, e0242192. [DOI: https://dx.doi.org/10.1371/journal.pone.0242192]

83. Santina, T.; Beaulieu, D.; Gagné, C.; Guillaumie, L. Using the intervention mapping protocol to promote school-based physical activity among children: A demonstration of the step-by-step process. Health Educ. J.; 2020; 79, pp. 529-542. [DOI: https://dx.doi.org/10.1177/0017896919894031]

84. Partridge, S.R.; Redfern, J. Strategies to Engage Adolescents in Digital Health Interventions for Obesity Prevention and Management. Healthcare; 2018; 6, 70. [DOI: https://dx.doi.org/10.3390/healthcare6030070]

85. Raeside, R.; Partridge, S.R.; Singleton, A.; Redfern, J. Cardiovascular Disease Prevention in Adolescents: eHealth, Co-Creation, and Advocacy. Med. Sci.; 2019; 7, 34. [DOI: https://dx.doi.org/10.3390/medsci7020034]

86. Ruan, S.; Raeside, R.; Singleton, A.; Redfern, J.; Partridge, S.R. Limited Engaging and Interactive Online Health Information for Adolescents: A Systematic Review of Australian Websites. Health Commun.; 2021; 36, pp. 764-773. [DOI: https://dx.doi.org/10.1080/10410236.2020.1712522] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31964190]

87. Eyles, H.; Jull, A.; Dobson, R.; Firestone, R.; Whittaker, R.; Morenga, L.T.; Goodwin, D.; Ni Mhurchu, C. Co-design of mHealth Delivered Interventions: A Systematic Review to Assess Key Methods and Processes. Curr. Nutr. Rep.; 2016; 5, pp. 160-167. [DOI: https://dx.doi.org/10.1007/s13668-016-0165-7]

88. Taggart, L.; Truesdale, M.; Dunkley, A.; House, A.; Russell, A.M. Health Promotion and Wellness Initiatives Targeting Chronic Disease Prevention and Management for Adults with Intellectual and Developmental Disabilities: Recent Advancements in Type 2 Diabetes. Curr. Dev. Disord. Rep.; 2018; 5, pp. 132-142. [DOI: https://dx.doi.org/10.1007/s40474-018-0142-5] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30148038]

89. Taggart, L.; Doherty, A.J.; Chauhan, U.; Hassiotis, A. An exploration of lifestyle/obesity programmes for adults with intellectual disabilities through a realist lens: Impact of a ‘context, mechanism and outcome’ evaluation. J. Appl. Res. Intellect. Disabil.; 2020; 34, pp. 578-593. [DOI: https://dx.doi.org/10.1111/jar.12826] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33342030]

90. Rütten, A.; Frahsa, A.; Abel, T.; Bergmann, M.; de Leeuw, E.; Hunter, D.; Jansen, M.; King, A.; Potvin, L. Co-producing active lifestyles as whole-system-approach: Theory, intervention and knowledge-to-action implications. Health Promot. Int.; 2019; 34, pp. 47-59. [DOI: https://dx.doi.org/10.1093/heapro/dax053]

91. Champion, K.E.; Gardner, L.A.; Mc Gowan, C.; Chapman, C.; Thornton, L.; Parmenter, B.; McBride, N.; Lubans, D.R.; McCann, K.; Spring, B. et al. A Web-Based Intervention to Prevent Multiple Chronic Disease Risk Factors Among Adolescents: Co-Design and User Testing of the Health4Life School-Based Program. JMIR Form. Res.; 2020; 4, e19485. [DOI: https://dx.doi.org/10.2196/19485]

92. Peiris-John, R.; Dizon, L.; Sutcliffe, K.; Kang, K.; Fleming, T. Co-creating a large-scale adolescent health survey integrated with access to digital health interventions. Digit. Health; 2020; 6, pp. 1-13. [DOI: https://dx.doi.org/10.1177/2055207620947962]

93. Hardt, J.; Canfell, O.J.; Walker, J.L.; Webb, K.-L.; Brignano, S.; Peu, T.; Santos, D.; Kira, K.; Littlewood, R. Healthier Together: Co-design of a culturally tailored childhood obesity community prevention program for Maori & Pacific Islander children and families. Health Promot. J. Aust. Off. J. Aust. Assoc. Health Promot. Prof. 2021, 32 (Suppl. 1), 143–154.

94. Parder, M.-L. Possibilities for Co-Creation in Adolescents’ Alcohol Prevention. J. Creative Commun.; 2020; 15, pp. 147-164. [DOI: https://dx.doi.org/10.1177/0973258620924950]

95. Fennessy, G.; Burstein, F. Role of Information Professionals as Intermediaries for Knowledge Management in Evidence-Based Healthcare. Healthcare Knowledge Management; Springer: Berlin/Heidelberg, Germany, 2007; pp. 28-40.

96. Murphy, C.; Thorpe, L.; Trefusis, H.; Kousoulis, A. Unlocking the potential for digital mental health technologies in the UK: A Delphi exercise. BJPsych Open; 2020; 6, e12. [DOI: https://dx.doi.org/10.1192/bjo.2019.95]

97. Schuler, D.; Namioka, A. Participatory Design: Principles and Practices; CRC Press: Boca Raton, FL, USA, 1993.

98. Blomkamp, E. The promise of co-design for public policy. Aust. J. Public Adm.; 2018; 77, pp. 729-743. [DOI: https://dx.doi.org/10.1111/1467-8500.12310]

99. Ewert, B.; Evers, A. An Ambiguous Concept: On the Meanings of Co-production for Health Care Users and User Organizations?. VOLUNTAS Int. J. Volunt. Nonprofit Organ.; 2012; 25, pp. 425-442. [DOI: https://dx.doi.org/10.1007/s11266-012-9345-2]

100. Palumbo, R.; Manna, R. What if things go wrong in co-producing health services? Exploring the implementation problems of health care co-production. Policy Soc.; 2017; 37, pp. 368-385. [DOI: https://dx.doi.org/10.1080/14494035.2018.1411872]

101. Israel, B.A.; Schulz, A.J.; Parker, E.A.; Becker, A.B. REVIEW OF COMMUNITY-BASED RESEARCH: Assessing Partnership Approaches to Improve Public Health. Annu. Rev. Public Health; 1998; 19, pp. 173-202. [DOI: https://dx.doi.org/10.1146/annurev.publhealth.19.1.173]

102. Baum, F.; MacDougall, C.; Smith, D. Participatory action research. J. Epidemiol. Community Health; 2006; 60, 854. [DOI: https://dx.doi.org/10.1136/jech.2004.028662]

103. Slattery, P.; Saeri, A.K.; Bragge, P. Research co-design in health: A rapid overview of reviews. Health Res. Policy Syst.; 2020; 18, 17. [DOI: https://dx.doi.org/10.1186/s12961-020-0528-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32046728]

104. Oliver, K.; Kothari, A.; Mays, N. The dark side of coproduction: Do the costs outweigh the benefits for health research?. Health Res. Policy Syst.; 2019; 17, pp. 1-10. [DOI: https://dx.doi.org/10.1186/s12961-019-0432-3]