Evaluation of parent and youth experiences in advisory groups as part of a mental healthcare clinical trial: protocol for a mixed-method study

Abstract

Introduction

Patient engagement in healthcare research is a necessity to ensure that research objectives align with priorities, outcomes and needs of the population under study, and to facilitate ease of implementation and adoption of findings. In clinical trials, there is an increasing focus on patient engagement during the planning and conduct of clinical trials due to the potential for ethical and methodological benefits. As patient engagement in clinical trials increases, there is a need to evaluate the approaches of these activities to contribute evidence on what is most appropriate and successful. The purpose of this study is to evaluate patient engagement processes and the activities of patient partners during and after a paediatric mental healthcare trial.

Methods and analysis

Using a mixed-methods study design, we will evaluate patient partners’ engagement activities across set time-points during the trial and after trial completion. In this study, the term ‘patient partner’ is inclusive of two groups of people with lived experience: (1) caregivers (parents, formal/informal caregivers and family), and (2) youth (aged 15–24 years). Engagement will be evaluated using the participant and project questionnaires of the Public and Patient Engagement Evaluation Tool (PPEET), followed sequentially by semi-structured interviews. Quantitative data from the PPEET questionnaire will be analysed and reported using descriptive statistics. Data from open-ended questions from the PPEET questionnaires and semi-structured interviews will be analysed using thematic analysis.

Ethics and dissemination

Approval from Athabasca University Research Ethics Board will be obtained for this project. Findings will be disseminated at both academic and public venues whether in-person or online, and using platforms that are caregiver and youth friendly.

Trial registration number

NCT04902391.

Correspondence to Jananee Rasiah; [email protected]

Strengths and limitations of this study

• Validated evaluation tool used to measure outcomes of patient engagement across a paediatric mental healthcare trial.

• Semi-structured interviews to elicit patient engagement experience.

• Dedicated resources for patient engagement from concept to end-of-grant knowledge translation activities.

• Limitation includes the ability to sustain parent and youth engagement over the course of the trial.

Introduction

Within the last decade, research funding organisations in the UK, the USA, Canada, Australia and New Zealand have required that patient engagement be an integral part of the design, conduct, dissemination and implementation of findings emerging from social and healthcare research.^1–5 Patient engagement occurs when patients become partners through meaningful and active collaboration across the research process.^{2 6} A recent systematic review has reported ethical, methodological and study quality benefits from engaging patients as partners in health research.⁶ These benefits have also been demonstrated through youth engagement in research, in particular, in the areas of health promotion,⁷ mental health,^{8 9} chronic pain¹⁰ and community development.¹¹ Steps taken to recognise youths’ expertise within engagement practices in research are equally beneficial and include: (i) ensuring a youth friendly approach (eg, listening to what youth have to say and asking questions in a non-judgemental manner); (ii) recognising diversity among youth (eg, recruiting youth from different developmental stages and with varied sociodemographic characteristics); (iii) formalising recognition of contributions that are authentic for youth (eg, providing compensation for engagement, writing references for school/university/jobs, offering coauthorship on findings or providing certificates of achievement) and (iv) establishing youth-friendly environments (eg, creating respectful and welcoming spaces for ongoing communication among youth, adult researchers and adult patient partners whether virtual or in-person and ensuring support for youth who may find research topics to be potentially triggering).¹² When youth have been engaged in research with these practical recommendations in place, the experiences are youth-friendly and the results are more likely to be implemented and adopted widely.^12–14

There is the emerging evidence around the benefits to engage patients as partners in clinical trials aimed to evaluate non-pharmacological or pharmacological interventions.^15–18 These benefits include: (i) more appropriate and sensitive/ethical research designs; (ii) more appropriate wording and timing of administration of research instruments and interventions; (iii) improved readability and accessibility of research materials (ethical benefits); (iv) more patient and/or caregiver relevant research outcomes or end points (methodological benefits); (v) improved recruitment, retention, diversity and trial experience/satisfaction of study participants; (vi) better adherence to the trial protocol and (vii) faster study completion (study quality benefits).¹⁸ Given this evidence, we saw the benefits of prioritising engagement with patients as partners with lived experience comprised youth and caregivers (discussed in more detail below in the section: Patient Engagement Plan) across the lifespan of our paediatric mental healthcare trial.

Overview: mental healthcare paediatric clinical trial

Trial development began in 2018, when members of our research team alongside a group of patients, families and healthcare providers identified the need to make improvements to paediatric mental healthcare in emergency departments (EDs).¹⁹ This group recognised EDs as a vital clinical care setting for children and youth experiencing acute mental health or substance misuse crises.²⁰ Such crises were acknowledged to be stressful and overwhelming, leaving the youth and family vulnerable in the absence of appropriate care. The team reinforced the position that ED-based care should be: (1) family centred and multidisciplinary, (2) informed by evidence-based approaches that identify risk, inform ED care and disposition decision-making and (3) consider family needs and preferences.^{20 21} In response to this position, a novel ‘bundle’ of mental healthcare was created with feedback from patient partners (comprised separate groups of youth and parent/caregivers). The bundle components were discussed and prioritised through several in-person meetings and teleconferences that took place between 2017 and 2019.²² Patient partners also prioritised study outcomes and selected outcome measures that were most relevant for self-reporting in mental health, satisfaction with care and family functioning. The results of this engagement were applied to both the trial as well as quasi-experimental study that provided pilot data to inform the trial.²² The bundle combines three evidence-based approaches to ED care, as follows:

• Ask Suicide-Screening Questions to identify suicide risk at ED triage and to facilitate faster access to appropriate treatment pathways.^23–25

• A clinical mnemonic (Home, Education, Activities/peers, Drug/alcohol, Suicidality, Emotions and behaviour, Discharge Resources), for a focused mental health psychosocial evaluation to guide ED care and disposition decision-making.^{21 26}

• A ‘choice’ appointment post-ED care, in accordance with the Choice And Partnership Approach to care. This is a family centred approach to mental health service organisation,²⁷ and its use recognises the value of connecting children and youth to follow-up services after a crisis.

The bundle will be evaluated using a type 1 comparative effectiveness-implementation hybrid design.²⁸ A cluster randomised trial will evaluate the effectiveness of the bundle compared with current standards of ED mental healthcare, while bundle implementation will be evaluated using multimethods to better understand barriers and facilitators to the adoption in ED care. The trial will take place in eight paediatric EDs across Canada in partnership with Pediatric Emergency Research Canada (PERC), a national research network. The Conjoint Health Research Ethics Board (CHREB), University of Calgary approved this study effective 4 January 2021 (reference ID: REB20-1825). This trial was registered at clinicaltrials.gov (Identifier: NCT04902391) with an estimated start date of 1 November 2021 and completion date of 1 August 2025. The protocol for this trial was submitted elsewhere for publication (see online supplemental appendix A for trial timeline).

A patient engagement plan was created for the trial that was informed by principles of patient engagement from many sources, including those specific to youth and caregivers.^{2 12 29–32} The plan will be evaluated in a trial substudy, which is described in ‘Patient engagement evaluation study’ section. The purpose of the patient engagement evaluation study is twofold¹: (1) evaluate the engagement of youth and caregivers in the trial design, conduct, outcomes (and knowledge translation of outcomes) of the trial and (2) obtain research team members’ perspectives on the impact of patient engagement on all stages of the trial.

Patient and public involvement

The study protocol was conceptualised by JR (lead for patient engagement evaluation study) with support from AN (co-principal investigator of the primary mental healthcare paediatric trial). LM (caregiver lead) and MP (youth lead) with lived experience and extensive experience in patient engagement reviewed and provided feedback on this study protocol. According to the patient engagement plan described below, all caregiver and youth advisory members will be invited to provide feedback on both process and findings at different time-points across the primary mental healthcare paediatric trial. Feedback by advisory group members on both trial process and findings will be shared at the executive research team meetings for discussion and deliberation. Postexecutive research team meetings, a brief written report about how suggestions have been incorporated will be circulated to advisory group members.

For the patient engagement evaluation study described below, caregiver and youth advisory group members will be invited to partake in the interpretation of the aggregate results, editing and preparation of the manuscript for publication. Caregiver and youth advisory group members will be invited to contribute to the overall knowledge integration and dissemination plan. Both patient partner contributors (LM and MP) met the ICMJE criteria for authorship and as such we acknowledge their valuable contributions through coauthorship of this study protocol. Engagement of patient partners has ensured further accuracy, readability and relevance of this protocol to the science and practice of patient engagement in paediatric clinical trials.

Patient engagement plan

Overview

The patient engagement plan has two foci: (i) consultation in trial planning, whereby youth and caregivers had the opportunity as advisors to provide feedback on the trial design and (ii) partnership, whereby youth and caregivers will be partners alongside research team members during trial conduct, with both groups actively involved in collaborating and leading trial-related activities. These foci align with the practices recommended by the Canadian Institute of Health Research when developing partnerships between patients and researchers.³³ We will use the term ‘patient advisor’ to denote those who are involved in a one-time engagement activity versus ‘patient partner’ who will provide feedback during patient engagement activities throughout the lifespan of the trial.

Participants and recruitment

We used a multipronged approach to recruit patient partners representing two groups of people with lived experience: (1) caregivers (including parents from diverse family forms), formal/informal caregivers and family and (2) youth (aged 15–24 years). We will specify the group we refer to within the protocol for particular instances as needed, but otherwise we mean both groups. Initial recruitment began in 2017 with youth and caregivers from two family advisory councils for child and adolescent mental health services in Calgary and Edmonton, Alberta, Canada invited to provide feedback and perspectives on bundle design, trial planning and grant submission. The participation of these advisors has concluded, with one partner staying as a team member for the trial, since funded.

The next phase of recruitment will focus on the participation of youth in a Youth Advisory Group and caregivers in a Caregiver Advisory Group; the groups will be active during trial conduct and post-trial knowledge translation activities. During youth and caregiver recruitment, we will prioritise equity, diversity and inclusion-based practices in patient engagement,³⁴ and plan to recruit individuals from different demographic groups, based on age, gender identity, cultural identity, disability or education. We will set up two advisory groups to minimise power differentials among youth and caregivers and foster a safe space, whereby all members are comfortable sharing their perspectives.

Youth engagement liaisons/coordinators from the Centre for Addiction and Mental Health (https://www.camh.ca/) (led by MP) will assist with recruitment of youth to join the Youth Advisory Group. Caregiver recruitment will be overseen by JR . Convenience sampling will be used to recruit individuals through organisations who partnered with the team (Canadian Mental Health Association, Translating Emergency Knowledge for Kids (TREKK), and Children’s Healthcare Canada).

Youth and caregivers who indicate interest in participating will be invited to attend a virtual introductory meeting (one for youth; one for caregivers) with the trial co-leads (SF and AN) and patient engagement leads (JR, MP and LM). The meeting will provide an opportunity for individuals to learn about the trial and the goal of patient partner engagement. Those interested in joining the team as advisory group members will be asked to contact the trial’s patient engagement evaluation study lead (JR) and youth engagement lead (MP) to further discuss and confirm their interest. Informed consent will be obtained at that time. Patient partners can voluntarily revoke consent at any time during the course of the trial. Patient partners will be supported by the patient engagement leads throughout the course of the project, as described further under advisory group activities. The goal is to have membership in both advisory groups filled between November 2021 and January 2022; the trial begins February 2022.

Sample size

We could not find evidence in the literature to support best practice for optimal group size for patient engagement in research. Therefore, we will exercise pragmatic considerations and recommendations from team members and project liaisons with extensive experience working in patient engagement. In particular, the size of the group should foster meaningful engagement, so that we are able to have robust discussion, and patient partners can share ideas in a safe space. Conversely, we do not want a group too large that it is ineffective to engage partners in discussions. We anticipate a target size of 5–10 patient partners per advisory group will be sufficient to lead to a meaningful discussion.

Patient partners that do not continue for the full duration of planned patient engagement activities will be included in the sample if they are willing. Patient partners who attend the virtual introductory meeting with the study co-lead, but later decline to participate in an advisory group will be asked if they are willing to share confidentially the reasons that led to the decision to forego participation. If we have attrition of advisory group members at any stage of the engagement process, we want to have the opportunity to formally document their reasons that led to that decision as part of the evaluation of patient engagement.

Advisory group activities

Advisory groups will be co-chaired by MP, LM and JR. At the first meeting of each group, members will discuss group norms, establish terms of reference for the respective advisory groups, including role description and individual goals as they relate to the patient engagement activities. Meetings will occur at least four times per year for up to 3 years, which is aligned with key milestones within the trial (see online supplemental appendix B). Compensation will be provided at a set annual rate per year that was budgeted in the grant. Institutionally approved videoconferencing platforms will be used to exchange and share information about the study. Patient partners will have the ability to choose the types of engagement activities that they would like to participate in, and this will be determined on an individual and/or advisory group basis. These activities include co-leading the advisory group alongside the chairs, providing feedback on recruitment materials for the trial and telephone scripts used during data collection with trial participants, informing changes to recruitment and retention processes during trial conduct, providing impressions on the results and designing/preparing materials to disseminate trial results (eg, statements for social media use, infographics for families). Patient partners will also have the opportunity to apply for funding to attend conferences and courses related to children’s mental health (budgeted within the trial) and/or work with the patient engagement evaluation study lead (JR) to plan and conduct evaluation meetings.

Patient engagement evaluation study

Design

We are using a mixed-method design to evaluate the trial’s patient engagement plan.^{35 36} In particular, we chose the exploratory sequential mixed-method design because we will first complete data collection and analyses using quantitative methods followed by qualitative methods.³⁷ This will involve surveying patient advisors and partners as well as research team members on engagement experiences followed by semi-structured interviews of patient partners to gather more detailed information on engagement experiences and activities highlighted in survey responses. In addition, we will place equal weighting on both quantitative and qualitative methods³⁷ because our research purpose is to evaluate engagement and determine impact of that engagement to the paediatric mental healthcare trial. Thus, to do that effectively, we emphasise that the data collected and analysed using both methods will address the twofold purpose. We will use the Good Reporting of Mixed Method Study Criteria³⁸ and the Guidance for Reporting Involvement of Patients and the Public³⁹ to assess the quality of reporting of our mixed-method patient engagement evaluation study procedures and findings.

Framework

We are using a framework to guide the evaluation. This framework contains four domains—integrity of design and process; flexibility; mentorship; influence and impact—and was adapted from the published literature on patient and youth engagement.³²⁴⁰ Integrity of design and process will be assessed by examining advisory group representation (eg, Do members represent different experiences based on age, gender identity, cultural identity, disability or education?) and how group members are supported in patient engagement activities (eg, Are members compensated for their involved in activities? Is information produced at an appropriate education level for members?).³² Flexibility of advisory groups with caregivers and youth will be assessed by examining the terms of reference (role, structure, norms and frequency of interactions) of the groups and how those terms of reference are carried out across the research project.⁴⁰ Mentorship will be assessed by asking patient partners about advisory group trial activities and opportunities that aligned with personal goals.⁴⁰ Influence and impact will be assessed by examining whether patient engagement activities informed or changed trial decisions.³²

Methods

Engagement Evaluation Tool

We will use the Public and Patient Engagement Evaluation Tool (PPEET) to evaluate patient engagement across our domains of interest: integrity of design and process, flexibility, mentorship and influence and impact. The PPEET comprises two questionnaires which will allow us to evaluate perspectives and experiences of advisory group members (participant questionnaire) and research team members (project questionnaire). The participant questionnaire also contains questions to evaluate one-time and ongoing patient engagement experiences, which will ensure that patient partners with different experiences contribute to the evaluation. At this time, we are aware of at least 27 tools to evaluate patient and public engagement.^{1 41 42} We chose the PPEET because it was used to evaluate patient and public engagement within the context of health research. The PPEET was highly rated by the Centre of Excellence on Partnership with Patients and the Public because of high scores in the domains of scientific rigour, patient and public perspective, comprehensiveness and usability.^{1 41 42}

In table 1, we outline how our domains of interest will be assessed using the PPEET participant and project questionnaires. We also identify time-points of when each questionnaire will be administered. Advisory group members will be asked to fill out a six-item demographic questionnaire, so we can assess diversity of perspectives, as part of integrity of design and process. The questions include year of birth, sex, group of people or communities one identifies with, highest level of education completed, current work status and paid experience in a healthcare profession. Additionally, we will include a description about advisory members’ previous experience with patient engagement in research (information will be obtained when we recruit members).

Table 1

Overview of the approach to measuring patient engagement

Integrity of design and process; flexibility; mentorship
PPEET questionnaire	Questionnaire outcomes	Administration time-point
Participant questionnaire: one-time engagement activities (module A—parts A and B)	Advisory group members represent diverse range of views. Advisory group members are provided support that enable participation in engagement activities. Online platforms suitable for communication with patient partners. Clear and bidirectional communication achieved among patient partners and researchers. Mentorship opportunities created and tailored to patient partners’ interests.	At the end of the activity on the same day
Participant questionnaire: ongoing engagement activities (module B—parts A and B)		Between 3 and 6 months
Project questionnaire (module A—parts A, B and C)		Pre-trial conduct (pre-recruitment of trial participants)
Influence and impact
Participant questionnaire: one-time engagement activities (module A—parts C and D)	Engagement activities inform planning and/or decision making in the trial. Engagement activities improve patient partners’ knowledge of mental health and substance use crises in emergency departments. Engagement activities improve patient partners’ knowledge of the patient and family centred acute mental healthcare bundle and need for implementation. Increased confidence and trust of patient partners in members of the team. Increased confidence of patient partners in the study purpose. Increased confidence of patient partners in the impact of bundle implementation on mental health.	At the end of the activity on the same day
Participant questionnaire: ongoing engagement activities (module B—parts C and D)		Between 3 and 6 months
Project questionnaire (module B—parts A, B and C)	Clinicians, staff and researchers improve knowledge on patient partners’ perspectives about the bundle.	Post activity
Project questionnaire (module C—parts A, B and C)		3–6 months after engagement has concluded

PPEET, Public and Patient Engagement Evaluation Tool.

PPEET participant questionnaire

The one-time engagement questionnaire comprises 19 questions: 13 questions based on a five-point Likert scale (range: strongly disagree to strongly agree) and six open-ended questions. The questionnaire will be administered to each patient advisor immediately after the conclusion of an engagement activity. The ongoing engagement questionnaire will be administered to patient partners every 3–6 months during trial conduct. This questionnaire comprises a total of 20 questions: 14 questions based on a five-point Likert scale (range: strongly disagree to strongly agree) and six open-ended questions.

PPEET project questionnaire

We will use the PPEET three-module project questionnaire to assess involvement in patient engagement among the principal investigators, project manager, implementation site leads and youth engagement liaisons/coordinators. The first module (module A) consists of 12 questions on a five-point Likert scale (range: strongly disagree to strongly agree) and three open-ended questions to assess how and whether study team members have chosen appropriate patient engagement activities during the planning stage. This module will be administered prior to trial conduct and after the run-in period. The second module (module B) identifies how well patient engagement activities were executed (10 five-point Likert questions; six open-ended questions) and will be administered after the trial engagement activities are completed. The third module (module C) will evaluate the impact of patient engagement activities. This module includes four questions (five-point Likert scale; range: strongly disagree to strongly agree) and four open-ended questions, which will be administered 3–6 months after the completion of the engagement activities.

Semi-structured interviews

We will interview patient partners at the end of the trial after the completion of all engagement activities. Interview questions will be developed based on the themes that emerge from the open-ended responses collected at different time-point using the PPEET with the intent to gather more information on engagement experiences and activities.⁴³ Patient partners will be offered the choice to participate in individual or focus group interviews that will be conducted by a research nurse/graduate student. Focus group interviews will be held separately for youth and caregivers. A final forum inviting all patient partners and research team members to attend will be held to discuss and obtain feedback on synthesised questionnaire results and interview findings.

Additional data sources: supporting documents and trial protocol documents

Supporting documents for the paediatric mental healthcare trial such as meeting agendas, minutes or tailored resources for advisory group members will be reviewed monthly and will serve as additional data sources to understand patient engagement activities. Trial protocol documents will be collected annually from the trial research coordinator to determine how feedback from advisory group members were used to inform/modify trial decisions.

Data collection and management

We will administer the PPEET online using the Research Electronic Data Capture (REDCap) tools⁴⁴ hosted and supported by the Women and Children’s Health Research Institute at the University of Alberta. Data collected through this platform by a research nurse/graduate student will be de-identified and will not be tracked to any individual participant. In exceptional cases, such as limited participant access to online environments, the PPEET will be administered via telephone with answers entered by the interviewer into the REDCap database. Data will be stored in REDCap until exported for analysis.

Semi-structured interviews will be conducted via telephone/videoconferencing platform hosted by the University of Calgary (Zoom) and the audio will be recorded. Interview recordings will be transcribed verbatim, checked for accuracy against the original recordings and identifiers will be removed from the transcripts to maintain confidentiality. Data from the semi-structured interviews and the audio recordings will be stored securely in a server at the University of Alberta until the data are transcribed and subsequently analysed.

Field notes will be kept when additional data sources are reviewed. Supporting documents will be reviewed during regular advisory group meetings to keep informed of the trial progress. Trial protocol documents will be collected and reviewed annually from the trial research coordinator to note modifications that have been implemented across sites based on advisory group members’ feedback.

Data analysis

Data from the PPEET will be entered into SPSS V.27. We will use descriptive statistics (eg, means and SD, medians with IQR) to report group characteristics and responses to the Likert scale questions. Frequency and percentages will be reported for categorical information. Data from open-ended questions⁴⁵ and semi-structured interviews will be imported into NVIVO V.12. We will undertake thematic analysis⁴⁵ to analyse these data. Data from open‐ended questions will be grouped by question and then coded for common themes that arise across the data. The primary coder (research nurse or graduate student) and first author (JR) will meet at least three times during the development and application of the coding scheme as part of an iterative process to ensure coding reliability. Coding reliability by the study co-lead (AN) will be completed to resolve inconsistencies or disputes and to review independent coding of selected data excerpts.

Data from semi-structured interviews will be analysed using the coding scheme as a guide. Data that cannot be coded to existing codes or themes will be noted. In the later stages of analysis, the study co-lead (AN) will independently code data excerpts to maintain coding reliability. Advisory group members’ feedback on supporting documents and trial protocol documents will be reviewed and synthesised into field notes by a research nurse and research assistant, which will be reported as a narrative.

Ethics and dissemination

Ethical approval for this study was obtained through the Athabasca University Research Ethics Board (file no: 24575) for 1 year and is subject for renewal on an annual basis since the project is ongoing beyond 1 year. All participating advisory group members will receive an information sheet that will provide details on the purpose of the study, identify the potential risks/benefits and explain the voluntary nature of their participation. Patient advisors and partners can revoke consent from participating in the parent or youth advisory group and evaluation activities at any time. Patient advisors and partners may choose to omit particular questions while filling out the survey or during the interview (pertains only to patient partners). All data will be de-identified; therefore, individual participant data cannot be removed once collected. Data will be kept confidential. All data will be stored using secured software on a password-protected server and device.

Patient engagement evaluation findings will be shared with parent and youth advisory group members during virtual meetings. Advisory group members will be asked about how the results fit within the organising framework and whether the patient engagement outcomes were met. Dissemination of findings from this patient engagement evaluation study for the paediatric mental healthcare trial will be in the form of an academic publication in a reputable peer-reviewed journal, presentations at TREKK and PERC conferences/meetings, public presentations at appropriate venues and as posts or blogs on the research study website and online platforms that are parent and youth friendly. Research team members and advisory group members will be invited to coauthor, co-develop and co-present these findings targeted at multiple venues.

Data availability statement

No data are available. Not applicable.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Twitter @JananeeRasiah, @MariaJ_Santana

Contributors I, the Submitting Author has the right to grant and does grant on behalf of all authors of the work (as defined in the below author licence), an exclusive licence and/or a non-exclusive licence for contributions from authors who are: (i) UK Crown employees; (ii) where BMJ has agreed a CC-BY licence shall apply and/or (iii) in accordance with the terms applicable for US Federal Government officers or employees acting as part of their official duties; on a worldwide, perpetual, irrevocable, royalty-free basis to BMJ Publishing Group Ltd ('BMJ') its licensees and where the relevant Journal is co-owned by BMJ to the co-owners of the Journal, to publish the Work in BMJ Open and any other BMJ products and to exploit all rights, as set out in our licence. The Submitting Author accepts and understands that any supply made under these terms is made by BMJ to the Submitting Author unless you are acting as an employee on behalf of your employer or a postgraduate student of an affiliated institution which is paying any applicable article publishing charge ('APC') for Open Access articles. Where the Submitting Author wishes to make the work available on an Open Access basis (and intends to pay the relevant APC), the terms of reuse of such Open Access shall be governed by a Creative Commons licence—details of these licences and which Creative Commons licence will apply to this work are set out in our licence referred to above. The study protocol was conceptualised by JR (lead for patient engagement substudy) with support from AN (co-principal investigator of the primary mental healthcare paediatric trial). LM (caregiver lead) and MP (youth lead) with lived experience reviewed and provided feedback on this study protocol. SF (co-principal investigator of the primary mental healthcare paediatric trial) critically reviewed the manuscript while considering the paediatric trial timeline. KP was instrumental in creating the first draft of the patient engagement plan and obtaining feedback about the patient engagement activities from patient partners. ME, YF, GH, M-JS, J-TF, AS, IJG, MS, AM, LK, BG, LP, SM, ELL, BW, GD, RP, KH and YNAH reviewed the manuscript and provided feedback on feasibility and timing of the patient engagement activities and planned evaluation methods.

Funding This trial is funded by the Canadian Institutes of Health Research (CIHR) as a Strategy for Patient-Oriented Research (SPOR) Innovative Clinical Trial Multi-Year Grant (funding reference number: MYG-171686).

Competing interests None declared.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the 'Methods and analysis' section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Boivin A, Richards T, Forsythe L, et al. Evaluating patient and public involvement in research. BMJ 2018; 363: k5147. doi:10.1136/bmj.k5147 http://www.ncbi.nlm.nih.gov/pubmed/30522999

² Canadian Institutes of Health Research. Patient engagement framework Strategy for Patient Oriented Research. Ottawa CIHR; 2014.

³ Frank L, Forsythe L, Ellis L, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research Institute. Qual Life Res 2015; 24: 1033–41. doi:10.1007/s11136-014-0893-3 http://www.ncbi.nlm.nih.gov/pubmed/25560774

⁴ Saunders C, Crossing S, Girgis A, et al. Operationalising a model framework for consumer and community participation in health and medical research. Aust New Zealand Health Policy 2007; 4: 13. doi:10.1186/1743-8462-4-13 http://www.ncbi.nlm.nih.gov/pubmed/17592651

⁵ Staley K. Exploring Impactimpact:Publicpublic involvement in NHS, public health and social care research; 2009.

⁶ McCarron TL, Clement F, Rasiah J, et al. Patients as partners in health research: a scoping review. Health Expect 2021; 24: 1378–90. doi:10.1111/hex.13272 http://www.ncbi.nlm.nih.gov/pubmed/34153165

⁷ Peréa FC, Sayles NR, Reich AJ, et al. "Mejorando Nuestras Oportunidades": Engaging Urban Youth in Environmental Health Assessment and Advocacy to Improve Health and Outdoor Play Spaces. Int J Environ Res Public Health 2019; 16. doi: doi:10.3390/ijerph16040571. [Epub ahead of print: 16 02 2019 ]. http://www.ncbi.nlm.nih.gov/pubmed/30781471

⁸ Bell E. Young persons in research: a call for the engagement of youth in mental health research. Am J Bioeth 2015; 15: 28–30. doi:10.1080/15265161.2015.1088977 http://www.ncbi.nlm.nih.gov/pubmed/26575810

⁹ Mawn L, Welsh P, Stain HJ, et al. Youth speak: increasing engagement of young people in mental health research. J Ment Health 2015; 24: 271–5. doi:10.3109/09638237.2014.998810 http://www.ncbi.nlm.nih.gov/pubmed/26193175

¹⁰ Birnie KA, Dib K, Ouellette C, et al. Partnering for pain: a priority setting partnership to identify patient-oriented research priorities for pediatric chronic pain in Canada. CMAJ Open 2019; 7: E654–64. doi:10.9778/cmajo.20190060 http://www.ncbi.nlm.nih.gov/pubmed/31699686

¹¹ Battaglia TA, Pamphile J, Bak S, et al. Connecting community to research: a training program to increase community engagement in research. Prog Community Health Partnersh 2019; 13: 209–17. doi:10.1353/cpr.2019.0021 http://www.ncbi.nlm.nih.gov/pubmed/31178456

¹² Hawke LD, Relihan J, Miller J, et al. Engaging youth in research planning, design and execution: practical recommendations for researchers. Health Expect 2018; 21: 944–9. doi:10.1111/hex.12795 http://www.ncbi.nlm.nih.gov/pubmed/29858526

¹³ Checkoway B, Richards-Schuster K. Youth participation in community evaluation research. American Journal of Evaluation 2003; 24: 21–33. doi:10.1177/109821400302400103

¹⁴ Hawke LD, Darnay K, Relihan J, et al. Enhancing researcher capacity to engage youth in research: researchers' engagement experiences, barriers and capacity development priorities. Health Expect 2020; 23: 584–92. doi:10.1111/hex.13032 http://www.ncbi.nlm.nih.gov/pubmed/32170902

¹⁵ Bartlett SJ, Barnes T, McIvor RA. Integrating patients into meaningful real-world research. Ann Am Thorac Soc 2014; 11 Suppl 2: S112–7. doi:10.1513/AnnalsATS.201309-327RM http://www.ncbi.nlm.nih.gov/pubmed/24559023

¹⁶ Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res 2014; 14: 89. doi:10.1186/1472-6963-14-89 http://www.ncbi.nlm.nih.gov/pubmed/24568690

¹⁷ Bagley HJ, Short H, Harman NL, et al. A patient and public involvement (PPI) toolkit for meaningful and flexible involvement in clinical trials - a work in progress. Res Involv Engagem 2016; 2: 15. doi:10.1186/s40900-016-0029-8 http://www.ncbi.nlm.nih.gov/pubmed/29062516

¹⁸ Vat LE, Finlay T, Jan Schuitmaker-Warnaar T, et al. Evaluating the "return on patient engagement initiatives" in medicines research and development: A literature review. Health Expect 2020; 23: 5–18. doi:10.1111/hex.12951 http://www.ncbi.nlm.nih.gov/pubmed/31489988

¹⁹ Bialy L, Plint AC, Freedman SB, et al. Pediatric emergency research Canada (PERC): Patient/Family-informed research priorities for pediatric emergency medicine. Acad Emerg Med 2018; 25: 1365–74. doi:10.1111/acem.13493 http://www.ncbi.nlm.nih.gov/pubmed/29873867

²⁰ Dolan MA, Fein JA, Committee on Pediatric Emergency Medicine. Pediatric and adolescent mental health emergencies in the emergency medical services system. Pediatrics 2011; 127: e1356–66. doi:10.1542/peds.2011-0522 http://www.ncbi.nlm.nih.gov/pubmed/21518712

²¹ Newton AS, Hartling L, Soleimani A, et al. A systematic review of management strategies for children's mental health care in the emergency department: update on evidence and recommendations for clinical practice and research. Emerg Med J 2017; 34: 376–84. doi:10.1136/emermed-2016-205939 http://www.ncbi.nlm.nih.gov/pubmed/28119350

²² Freedman S, Thull-Freedman J, Lightbody T, et al. Introducing an innovative model of acute paediatric mental health and addictions care to paediatric emergency departments: a protocol for a multicentre prospective cohort study. BMJ Open Qual 2020; 9: e001106. doi:10.1136/bmjoq-2020-001106 http://www.ncbi.nlm.nih.gov/pubmed/33318032

²³ Horowitz LM, Bridge JA, Teach SJ, et al. Ask Suicide-Screening questions (ASQ): a brief instrument for the pediatric emergency department. Arch Pediatr Adolesc Med 2012; 166: 1170–6. doi:10.1001/archpediatrics.2012.1276 http://www.ncbi.nlm.nih.gov/pubmed/23027429

²⁴ Horowitz LM, Wharff EA, Mournet AM, et al. Validation and feasibility of the ASQ among pediatric medical and surgical inpatients. Hosp Pediatr 2020; 10: 750–7. doi:10.1542/hpeds.2020-0087 http://www.ncbi.nlm.nih.gov/pubmed/32826283

²⁵ DeVylder JE, Ryan TC, Cwik M, et al. Assessment of selective and universal screening for suicide risk in a pediatric emergency department. JAMA Netw Open 2019; 2: e1914070. doi:10.1001/jamanetworkopen.2019.14070 http://www.ncbi.nlm.nih.gov/pubmed/31651971

²⁶ Cappelli M, Gray C, Zemek R, et al. The HEADS-ED: a rapid mental health screening tool for pediatric patients in the emergency department. Pediatrics 2012; 130: e321–7. doi:10.1542/peds.2011-3798 http://www.ncbi.nlm.nih.gov/pubmed/22826567

²⁷ York A, Kingsbury S. The choice and partnership approach: a transformational service model. United Kingdom: CAPA Publications, 2013.

²⁸ Curran GM, Bauer M, Mittman B, et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012; 50: 217–26. doi:10.1097/MLR.0b013e3182408812 http://www.ncbi.nlm.nih.gov/pubmed/22310560

²⁹ Amirav I, Vandall-Walker V, Rasiah J, et al. Patient and Researcher Engagement in Health Research: A Parent’s Perspective. Pediatrics. 2017; 140.

³⁰ Darnay K, Hawke LD, Chaim G. INNOVATE research. Toronto Youth Engagement Guidebook for Researchers; 2019.

³¹ International Association of Public Participation. The IAP2 public participation spectrum: IAP2, 2016. Available: http://www.iap2.org.au/resources/public-participation-spectrum

³² Abelson J, Li K, Wilson G, et al. Supporting quality public and patient engagement in health system organizations: development and usability testing of the public and patient engagement evaluation tool. Health Expect 2016; 19: 817–27. doi:10.1111/hex.12378 http://www.ncbi.nlm.nih.gov/pubmed/26113295

³³ Canadian Institute of Health Research (CIHR). Ethics guidance for developing partnerships with patients and researchers. Ottawa; CIHR 2020.

³⁴ Ontario HQ. Creating and sustaining patient and family advisory councils: Guides for common challenges - Recruiting for diversity. Ontario Health Quality Ontario; 2021.

³⁵ SAGE. Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks, California: SAGE Publications, Inc. 2, 2010. https://methods.sagepub.com/book/sage-handbook-of-mixed-methods-social-behavioral-research-2e.

³⁶ Morse JM. Approaches to qualitative-quantitative methodological triangulation. Nurs Res 1991; 40: 120–3. http://www.ncbi.nlm.nih.gov/pubmed/2003072

³⁷ Creswell JW, Clark VLP. Designing and conducting mixed methods research 2007.

³⁸ O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy 2008; 13: 92–8. doi:10.1258/jhsrp.2007.007074 http://www.ncbi.nlm.nih.gov/pubmed/18416914

³⁹ Boote J, Wong R, Booth A. 'Talking the talk or walking the walk?' a bibliometric review of the literature on public involvement in health research published between 1995 and 2009. Health Expect 2015; 18: 44–57. doi:10.1111/hex.12007 http://www.ncbi.nlm.nih.gov/pubmed/23033933

⁴⁰ Heffernan OS, Herzog TM, Schiralli JE, et al. Implementation of a youth-adult partnership model in youth mental health systems research: challenges and successes. Health Expect 2017; 20: 1183–8. doi:10.1111/hex.12554 http://www.ncbi.nlm.nih.gov/pubmed/28295940

⁴¹ Boivin A, L'Espérance A, Gauvin F-P, et al. Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expect 2018; 21: 1075–84. doi:10.1111/hex.12804 http://www.ncbi.nlm.nih.gov/pubmed/30062858

⁴² Dukhanin V, Topazian R, DeCamp M. Metrics and evaluation tools for patient engagement in healthcare Organization- and system-level decision-making: a systematic review. Int J Health Policy Manag 2018; 7: 889–903. doi:10.15171/ijhpm.2018.43 http://www.ncbi.nlm.nih.gov/pubmed/30316241

⁴³ Corbin J, Strauss A, Thousand Oaks C 3 ed. Basics of qualitative research: techniques and procedures for developing grounded theory; 2008.

⁴⁴ Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–81. doi:10.1016/j.jbi.2008.08.010 http://www.ncbi.nlm.nih.gov/pubmed/18929686

⁴⁵ Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3: 77–101. doi:10.1191/1478088706qp063oa