Internet-based acceptance and commitment therapy programme ‘Happiness Mom’ for well-being: a protocol for a randomised controlled trial

Abstract

Introduction

This randomised controlled trial (RCT) aims to examine the effects of an internet-delivered acceptance and commitment therapy (iACT) programme (‘Happiness Mom’) on the psychological well-being of working mothers.

Methods and analysis

The target population of the RCT will be employed mothers with at least one preschool child. Participants who fulfil the study’s eligibility criteria will be randomly assigned either to an iACT intervention group (n=200) or to a wait-list control group (n=200). Participants in the intervention groups will be asked to complete the programme within 12 weeks of the baseline survey. The intervention programme contains eight modules based on ACT. Primary outcomes are six components of psychological well-being, based on Ryff’s theory. Secondary outcomes are intention to leave their job, work engagement, work performance, sick leave days, psychological distress, euthymia, positive emotions, job and life satisfaction, social support and parental burn-out.

Ethics and dissemination

Ethical approval for this study has been obtained from the Research Ethics Review Board of Graduate School of Medicine, the University of Tokyo (No. 2019134NI). If the intervention programmes are found to be significantly beneficial, the programmes can be made available for all working mothers with preschool children in Japan.

Discussion

This study will contribute to the development of an internet-based self-care programme that is effective, feasible, low cost and accessible to improve the well-being of working mothers.

Trial registration number

UMIN000039918.

Correspondence to Professor Norito Kawakami; [email protected]

Strengths and limitations of this study

• The effects of an automated internet-based acceptance and commitment therapy (ACT) programme on the psychological well-being of working mothers with preschool children will be examined.

• The ‘Happiness Mom’ programme is a newly developed programme that involves discussion with psychology experts and mothers, designed to contribute to participants’ low dropout and high engagement in the trial.

• Multimedia self-help programmes (eg, text exercise, audio) are expected to enhance the intervention effect on ACT and mindfulness.

• A limitation of this study is that all outcomes will be self-reported.

Introduction

Working mothers are a vital part of the workforce worldwide.¹ Stable employment for mothers is important in terms of female economic independence and child development.² Healthy maternal employment is also highlighted in the United Nation’s Sustainable Development Goals (3: promote well-being for all, 5: empower women and 8: promote productive employment).^3–5 However, working mothers are often under psychological and physical pressure from their many responsibilities, including domestic activities, childbearing and paid work.^{6 7} This double burden from work and family life and work–family conflict make the lives of working mothers difficult.^8–11 In sex-segregated cultures like Japan’s, women must shoulder their caretaking responsibilities.¹² As a result, about 47% of Japan’s working women give up their jobs after the birth of their first child.¹³ They are required to change their way of living, with some losing their mastery and autonomy as a result.

Poor mental health is a common health-related outcome of the conflicts experienced by working mothers.^14–16 Although maternal employment itself is beneficial for mental health,^17–19 working mothers’ dual burden and poor work–family balance could often lead to depression, anxiety and burn-out.^{8 9 15 20–22} Working mothers with conflicts reportedly have poor psychological well-being (PWB).²³ Women with young children reported poorer PWB than women with older children.²⁴ Moreover, parental tasks are known to be more challenging to PWB among women than men.²⁴ PWB presents the potential for people to ‘live well’ (self-realised, fully functioning, purposefully engaged) as defined by Ryff.^{25 26} PWB has also been associated with better health outcomes and longer survival.^27–35 Therefore, PWB can be an important mental health outcome, as well as negative emotions and distress. Intervention programmes should be developed and established to promote PWB as well as to prevent depression, anxiety and burn-out among working mothers.

Previous studies showed the effectiveness of several types of interventions in improving PWB, including well-being therapy,³⁶ third wave cognitive–behavioural therapy (CBT)³⁷ and positive psychology approaches.^{38 39} Acceptance and commitment therapy (ACT) is one of the third waves of behavioural therapies and involves mindfulness and values-based exercises to help people accept their lives and commit to moving forward in a valued life direction.^40–42 ACT interventions have been found effective in reducing the stress of mothers of children diagnosed with autism.^43–49

Time constraints and busy schedules make it difficult for working mothers to seek mental health support.⁵⁰ Internet-delivered psychosocial intervention is feasible, low cost, effective and accessible.^{51 52} A comparison of face-to-face and internet interventions revealed no differences in their effectiveness in treating common mental disorders.^{53 54} Web-based intervention may, thus, be promising for busy working mothers.⁵⁵ Several systematic reviews showed that internet-delivered ACT (iACT) has been effective in managing depression and anxiety and improving the quality of life, even with a small effect size, in the general and clinical population.^56–58 For the working population, randomised controlled trials (RCTs) showed the effectiveness of ACT-based intervention (combined iACT and face-to-face group sessions) for improving negative mood outcomes (eg, depression, burn-out) among employees with psychological symptoms.^59–62 One RCT showed the significant effectiveness of improving Ryff’s PWB at postfollow-up (Cohend’s d=0.32, p<0.01), but not at 6-month or 12-month follow-up.^{60 63} A prestudy and poststudy of group ACT at worksite also showed the improvement of PWB among university staff.⁶⁴ The RCT of iACT among community-dwelling people seemed to improve PWB among those who had low PWB.⁶⁵ However, no study has investigated the effect of fully automated iACT without face-to-face sessions for improving PWB among workers regardless of psychological symptoms.

The aim of this RCT is to examine the effectiveness of a new and fully automated iACT programme "Happiness Mom" on improving PWB at posttreatment and 6-month follow-up among working mothers with at least one preschool child. We also examine the acceptability, appropriateness and feasibility of implementing it.

Methods

Trial design

This study is a two-arm parallel-group non-blinded RCT. Participants will be randomly allocated 1:1 either to the intervention or to the control group after completing a baseline online questionnaire survey. This study’s 12-week iACT programme was provided to the participants in the intervention group. Online follow-up surveys will be conducted at three (immediately after the intervention) and 6 months after the baseline survey. This protocol has been written according to the Standard Protocol Items: Recommendations for Interventional Trials guidelines.⁶⁶ Data generated by this research that supports the main findings will be made available as soon as possible, wherever legally and ethically possible. The data from this trial will be made available on reasonable request.

Participants

This study is intended for working mothers with at least one preschool child. The participants will be recruited from several private companies and individual applications through internet advertisements in Japan.

All participants must meet the following eligibility criteria:

• Women over 20 years of age.

• Currently employed for more than 30 hours per week.

• Have at least one preschool age child.

• Can access the internet via a PC: personal computer, URL: uniform resource locator, ID: identifier.

• Not on maternity leave at baseline.

This study is ‘a universal prevention programme’; thus, the participants will be recruited regardless of psychological symptoms or current regular psychotherapeutic treatment.

Recruitment and procedure

A flow chart of how to progress from screening for eligibility to inclusion in the final analysis is given in figure 1. Regarding company-based recruitment, the researchers have created the application website for each company. The website will include a description of the study and an application form. The researchers will ask the company’s department of human resources or occupational health sector to email the URL of a website to the candidates. The company will provide information for all candidates who seem to meet the eligibility criteria. For individual-based recruitment, the other website will detect the means of recruitment (which company or individual) by researchers. The URL of the website for individual recruitment will be spread through social networking service (SNS) advertisements (eg, Facebook) and personal announcements from the researchers. Candidates will click on the URL and read a full explanation of the purpose and procedures of the study on the website. Those who are interested in participating will be asked to mark the consent option and type their name, phone number and email. This consent information will be sent to the research centre and preserved. Workers who are not interested in participating will be asked to leave the website. Subsequently, participants who submitted the consent information will receive an email containing the URL of the baseline survey. They will be required to click the URL and input their baseline information. After completing the baseline questionnaire, the participants will be randomly assigned either to the intervention or to the control group. The researchers will inform the participants of their group assignment via email. To avoid delay between the randomisation and the beginning of intervention, the researchers will assign the participants to a group (which automatically sets the schedule for opening the new modules) as soon as the baseline survey is completed. The researchers will email the iACT programme’s URL and an ID and password to the participants in the intervention group. They will be asked not to share their ID, password, and the iACT programme content. Those in the control group will receive the iACT programme after the 6-month follow-up.

Figure 1. Participants’ flow chart.

Intervention programme

An internet-based, fully automated, eight-module self-help ACT programme ‘Happiness Mom’ designed for working women with preschool children was developed (figure 2). The first author led and organised the specialist team to develop the programme. The researchers (NS, KT and YK), who are working mothers with preschool children, have extracted the essential concerns of the working women, including their experience in occupational health settings. Informal interviews were conducted with two working women with pre-school children about their daily stressors, feelings, thoughts, dreams and ambitions. Based on the self-help ACT literature,^{67 68} the programme has been drafted by using a typical case experienced by mothers. The ACT specialists (outside the research team) and clinical psychologist (YS) who had experienced holding a group ACT for mothers have revised the programme to follow the psychological framework of ACT. A psychiatrist (DN) and a clinical psychologist (KI) confirmed the relevance of the psychotherapy. Original illustrations have been inserted to suit the preferences of the target population and to make it visually easy to understand. In each model, a single working mother case with a 2-year-old boy (fictitious) is presented as an example of applying ACT skills to daily life. The eight modules are presented in a fixed order; one module is accessible per week, with four optional reading materials (table 1). Each module will take 15–30 min to complete. The modules followed the six core processes, ‘hexaflex,’ which produces psychological flexibility.^{41 42 69} The programme modules cover well-being education (module 1), acceptance and willingness (module 2), defusion (module 3), mindfulness and self-compassion (module 4), self-as-context (module 5), value (module 6), committed actions (module 7) and wrap up (module 8). The optional reading materials address (1) parenting skills based on Positive Parenting Programme (Triple P)⁷⁰; (2) couple therapy based on assertive communication; (3) relaxation skills and (4) how to contact mental health services. Thirteen working mothers with small children were recruited by snowball sampling. They attended a 1-hour semistructured interview conducted by the first author after completing the programme. The participants were asked about its feasibility, usability, adaptability (ie, timing of reminder, the day of opening the new module) and suggestions for improving the programme. Based on the results of the interviews, the researcher modified the programme to make it more relevant to working mothers of small children.

Figure 2. Screenshots of Happiness Mom.

Table 1

Contents of the iACT ‘Happiness Mom’ programme for working mothers with preschool child(ren)

Module	Key concept	Estimated time	Contents
1	Well-being education	15 min	Defining happiness with psychological well-being model Ineffective coping strategy Don’t think about your thoughts (exercise) Difference between ‘pain’ and ‘suffering’ Exploring internal barriers (ie, thoughts, feelings) to values-based actions (exercise)
2	Acceptance and Willingness	15 min	Experiential avoidance How to accept negative feelings and thoughts (audio) Formulating own willingness (exercise)
3	Defusion	10 min	Fusion with ‘Mind’ Cognitive defusion technique of acknowledging ‘I am having the thought that …’ Leaves on a stream (metaphor)
4	Mindfulness and Self-compassion	30 min	Mindful breathing (audio), meditation (audio), and eating and drinking. Self-compassion ‘self-hug exercise’. Self-compassionate body scan (audio).
5	Self as context	15 min	Conceptualised self Tug‐of‐war (metaphor) Observing communication from the sky (exercise).
6	Value	20 min	Passengers on a bus (metaphor) Think of your own epitaph. (exercise) Defining what values are Clarifying one’s values (eg, epitaph, value in seven domains) (exercise)
7	Committed actions	30 min	Difference of thigs can be controlled or not Defining effective and values-based goals (exercise) Making action lists and strategies against internal barriers (exercise) Psychological flexibility -break the pattern and expand variety of actions.
8	Wrap up	6 min	Psychological well-being model New strategy to grow yourself up
Optional sessions		5 min/content	Positive parenting programme (based on Triple P) and Parenting mindfulness. Couple therapy (based on assertive communication). Relaxation skill and behavioural activation techniques. How to contact to mental health service of third party.
Additional functions			Can review what they wrote on exercise in courses (note function) Can share their feeling or impression with all participants on forum Can ask questions to researchers on forum

Happiness Mom does not have homework and is fully self-guided. The modules are not mediated by a therapist. Although no personalised feedback is provided, the case in the contents is about a single working mother with limited support for raising her 2-year-old son, a scenario expected to be familiar to the participants.

Well-being education (module 1)

In this module, the six dimensions of Ryff’s PWB model are introduced to seek happiness.²⁶ The differences between pain (ie, the discomfort we feel in response to a real-life problem) and suffering (ie, struggle by expanding negative thoughts or feelings in mind) are illustrated.^{41 67} The exercise ‘Don’t think about your thoughts’ is provided.⁶⁷ Participants will perform three text exercises, in which they have to describe a painful experience, focusing on the reason for it (dysfunctional thoughts) and how to avoid such dysfunctional thoughts.

Acceptance and willingness (module 2)

In this module, experiential avoidance (attempting to avoid or control internal experiences) does not work.⁷¹ By contrast, acceptance, the adoption of an intentionally open, receptive, flexible and non-judgmental posture, is introduced as a new strategy. Participants will experience acceptance by following the audio guide and completing a text exercise about their willingness.

Defusion (module 3)

This module introduces the concept of cognitive fusion, or being ‘stuck’ to thoughts and self-evaluations. It encourages the practice of mind watching. Five exercises for defusion are offered: (1) ‘I think…’, (2) leaves on a stream (metaphor), (3) extracting the thoughts from your head and seeing them on the hand, (4) naming the mind and (5) making the endings of your thought funny. Participants will be encouraged to try defusion.

Mindfulness and self-compassion (module 4)

Mindfulness is the process of paying attention on purpose, in the present moment, and non-judgmentally.⁷² Three experiential exercises are introduced here: mindfulness breathing (3 min audio), mindfulness meditation (5 min audio) and drinking mindfulness. Self-compassion is presented as a healthy form of self-acceptance to treat oneself with warmth and understanding.^{73 74} Two writing exercises, ‘Be willing to give words of comfort and reassurance to yourself as if to a friend who is tired of painful feeling’ and ‘You can also write down some reassuring or relieving actions you can take to help yourself’ are included. The ‘self-hug’ is introduced with an explanation of oxytocin.^{75 76} Participants will try a self-compassion body scan (7 min audio) to recognise their sense of body and treat themselves with compassion.

Self as context (module 5)

The observing self is one facet of what ACT calls ‘self as context’. Self as context entails taking several perspectives on the self. In this module, conceptualised self and sticking are introduced with the tug-of-war metaphor.⁶⁷ Through writing exercises, participants will take a bird’s eye view of the situation.⁶⁸ Participants will be encouraged to acquire the ability to look inside themselves from ‘now’ and ‘here’.⁴¹

Value (module 6)

ACT defines values as desired global qualities of ongoing action.⁴¹ In this module, the metaphor of passengers on the ‘your life’ bus is explained. Values are the direction of the bus. Two text exercises encouraged participants to recognise one’s desire: (1) imagine what they would do if they had everything they wanted; (2) write their own epitaph. Thereafter, participants will write their values in seven important life domains (ie, work, leaning, parenting, relationship with partner, friends, hobbies and health) and rank the three values that are most important to them.

Committed actions (module 7)

Committed actions can be large or small moves in the direction of values. The difference between ‘what you can control’ and ‘what you cannot control’ is explained. This module emphasises the importance of focusing on actions as part of being in control. Participants will define several values-based goals, making action plans for each goal and measures for the internal barriers through text exercises. Some unfamiliar actions (eg, wearing different coloured socks) for breaking a pattern and expanding the variety of actions are introduced to improve psychological flexibility.

Wrap up-life is a journey: (module 8)

In this module, Ryff’s model of PWB is reviewed again. The importance of accepting fear, feeling and thoughts from a perspective of self-as-context with openness and willingness is emphasised. Participants are encouraged to break the cycle of experiential avoidance by taking committed action. After finishing this module, participants receive emails suggesting that they review the programme and their personal notes, and to continue taking the actions that they learnt in the programme.

Optional sessions

Four articles form the basis of optional sessions. Participants may read them at any time during the intervention. These articles address (1) parenting skills based on the Positive Parenting Programme (Triple P)⁷⁷; (2) the essence of couple therapy based on assertive communication; (3) relaxation skills (ie, 4-7-8 breathing,⁷⁸ progressive muscle relaxation⁷⁹) and behavioural activation techniques and (4) how to contact third-party mental health services.

Additional functions

Personal notepad

Participants’ text exercises will be stored in their ‘personal notepad’ for review at their convenience. If participants want to repeat an exercise, they can go from their notepad to find the exercise and rewrite it.

Online forum

The online forum is set for each module. Participants can use the forum to share their experiences, ask questions, and talk to staff with specialised knowledge about the programme. Participants can post on the board using their registered nicknames instead of their real names. The first author will respond to the comments and questions posted on the board.

Control group

Participants in the control group will receive the iACT programme after the 6-month follow-up. The participants in the intervention group and the control group can seek any mental health treatment as usual, such as stress management education or medical care, throughout the research period.

Outcomes

Table 2 summarises the outcome measures. Only the participants in the intervention group will be assessed regarding the process evaluation outcomes (eg, usability, satisfaction) at the 3-month follow-up. Information contamination will be assessed in the control group at the 3-month and 6-month follow-ups. All data will be collected using web-based self-report questionnaires. At the 3-month and 6-month follow-ups, the research centre will send at least two emails reminding non-respondents to complete the questionnaires.

Table 2

Assessment schedule of the outcome measures for the randomised controlled trial for ‘Happiness Mom’

Measurement	Aim	Baseline (T1)	3 months follow-up (T2)	6 months follow-up (T3)
Primary outcome
PWBS-42	Psychological well-being	X	X	X
Secondary outcomes
K6	Psychological distress	X	X	X
PBA-J	Parental burnout	X	X	X
UWES-9	Work engagement	X	X	X
HPQ	Job performance	X	X	X
Sick leave days	Sick leave days during the past 3 months	X	X	X
Intention to leave	Intention to leave their company/organisation	X	X	X
Satisfaction	Job and life satisfaction	X	X	X
Positive feeling	Positive feeling and adjective (hedonic well-being)	X	X	X
VAS	Perceived social support from the partner	X	X	X
MSPSS	Social support	X	X	X
Fear of COVID-19	Global fear and worry about COVID-19	X	X	X
Euthymia	Psychological flexibility and resilience	X	X	X
Process evaluation
IDMH	Acceptability, appropriateness, feasibility, satisfaction, adverse effect		X	X
Effect of COVID-19	Adverse effects of leaning by COVID-19
Others
Demographic data		X
Contamination	Contamination of information for a control group		X	X

HPQ, WHO Health and Work Performance Questionnaire; IDMH, Implementation outcome scale for Digital Mental Health; K6, Kessler’s Psychological Distress Scale; MSPSS, Multidimensional Scale of Perceived Social Support; PBA-J, Parental Burnout Assessment Japanese version; PWBS-42, Psychological well-being scale 42 items version; UWES-9, Utrecht Work Engagement Scale 9 items version; VAS, Visual Analogue Scale.

Primary outcome

Psychological Well-being (PWB;42 items)

PWB will be evaluated using the 42-item version of Ryff’s PWB Scales (PWBS). The PWBS originally consisted of six 7-item subscales for the assessment of six factors: (1) autonomy; (2) environmental mastery; (3) personal growth; (4) positive relations with others; (5) purpose in life and (6) self-acceptance.^{26 80} Response categories for these items are scored along a seven-point Likert scale ranging from strongly disagree (1) to strongly agree (7). The scores of some items will be reversed as recommended in Ryff’s original PWBS.^{26 80} The scores for six subscales will be calculated as averages; higher mean scores indicate greater PWB. The reliability and validity of the Japanese version of PWBS have been recently tested.⁸¹ The authors simplified the Japanese translations of the response options without changing their meaning.

Secondary outcomes

As secondary outcomes, these measurement scales will be used: psychological distress (K6), parental burn-out, work engagement, job performance, sick leave days, intention to leave, job and life satisfaction, positive feelings (hedonic well-being), perceived social support from the partner, social support, global fear of COVID-19 and euthymia. The details of these scales are available in online supplemental file.

Process evaluation

Implementation outcomes and adverse effects

For process evaluation, we will track each participant’s completion of each module. The researcher will take the 14 items of implementation outcomes for digital health interventions. This measurement is based on three important concepts: acceptability, appropriateness and feasibility.⁸² These concepts have been discussed with international specialists in implementation science and internet-based psychotherapy. Adverse effects (ie, harms) of eHealth interventions, such as physical symptoms (eg, tired eyes, stiff shoulders), mental symptoms (eg, insomnia) and dangerous experiences (eg, bumping into people while looking at a smartphone), will be covered in five items.

Contamination of information

To evaluate contamination of information among participants in a control group, participants will be asked at follow-up survey: ‘Have you got to know information on the contents of Happiness Mom from others in an intervention group?’

Demographic characteristics

Demographic data, such as age, marital status, household income, education, years in the workforce, occupations, industry, employment contract, company size, working hours per week, the number of children, disabled child, age and sex of the youngest child, pregnancy, and history of fertility treatments, will also be collected.

The impact of COVID-19

The researchers collected data about changes in working hours and working styles (eg, working from home, furlough) to capture the impact of COVID-19 on daily work, as the target population was recruited during the pandemic. In the 3-month follow-up, we will ask the intervention group, ‘Is there anything in the programme that you were unable to put into practice because of COVID-19?’

Sample size calculation

Sample size was calculated (total N=398) for the primary outcome to detect an effect size of 0.25 (Cohen’s d) with a statistical power of (1 – b)=0.80 in a two-tailed test (p<0.05). Estimated effect size was decided based on the subgroup analysis of a previous meta-analysis that examined the effect of internet-based CBT for improving well-being (Hedge’s g=0.25).⁵¹ Regarding this calculation, the researchers set the estimated sample size as N=400.

Randomisation

Participants who meet the eligibility criteria will be randomly allocated to the intervention group or control group. Participants will be stratified into two groups according to the score of K6 (4 or less, or 5 or more) and recruitment entry (company-based or individual) on the baseline survey. In addition to the analysis of the whole sample (to examine the universal intervention effect), we will analyse data by prespecified subgroups (to examine the selective intervention effect). Using a computer-generated random allocation sequence, an independent biostatistician has created a stratified permuted-block random table. The block size of this RCT will be fixed at 4. The stratified permuted-block random table will be password protected and blinded to the researchers. Only the research assistant will have access to it during the random allocation.

Duration

This programme is scheduled to start recruiting in April 2020 and will run until the end of June. The research will end in January 2021 with a follow-up assessment. In March 2021, the whole programme will be presented to the control group.

Statistical analysis

For the main pooled analysis, a mixed model for repeated measures conditional growth model analysis with an unstructured covariance matrix will be conducted using a group (intervention and control)*time (baseline, 3 months and 6 months follow-ups) interaction as an indicator of intervention effect. The primary analysis will follow the intention-to-treat (ITT) principle. This means that the participants will remain in the group to which they were randomised and not analysed according to the interventions actually received. Cohen’s d between groups will be calculated at each assessment term. Secondary outcomes will be analysed in the same way as the primary outcome. The statistical significance for all analyses in this study will be set as 0.05 (two tailed), and 95% CIs will be calculated.

For sensitivity analysis, a similar mixed model for repeated measures, but using the analysis of variance model, with an unstructured covariance matrix will be conducted. Missing values will be imputed by applying the maximum likelihood estimation using the mixed procedure. We will calculate Cohen’s d among participants excluded ‘not started’ subjects, following the modified ITT principle.

For subgroup analysis, we will use the stratification factor (ie, participants who scored 4 or less/5 or more in K6 at the baseline survey or recruitment method (company or individuals)) and analyse the results according to the prespecified subgroups. The Japanese version of SPSS V.26.0 (IBM) will be used.

Monitoring

As we anticipate no potential harm from this intervention, there will be no data monitoring committee, interim analyses or stopping rules.

Data availability

Deidentified individual participant data will be available on request after the main analysis paper has been published. Please email requests to the corresponding author.

Ethics and dissemination

Ethical and safety considerations

Ethical approval for this study has been obtained from the Research Ethics Review Board of Graduate School of Medicine, the University of Tokyo (No. 2019134NI). Participants will be fully informed that they may withdraw from the trial at any time by contacting the researchers. Any protocol modifications will be communicated to the institutional review board.

Data confidentiality

The collected data will be stored as linkable anonymising data. The principal investigator will retain access to the final dataset after the trial and assume responsibility for data integrity and the accuracy of analysis.

Dissemination and implementation plan of research findings

The findings of this study will be published in peer-reviewed international journals. They will also be presented at research conferences, academic symposia and seminars. The principal investigator will be listed as the corresponding author, and the authorship eligibility will conform to the International Committee of Medical Journal Editors. If the intervention programmes are found to be significantly beneficial, the programmes can be made available for all working mothers in Japan with preschool children.

Patient and public involvement

The first author had informal discussions with two female workers who were mothers of preschool children. Based on these conversations, three female researchers (NS, KT and YK), who were working mothers with preschool children, modified the contents to reflect a real situation. One patient and public involvement (PPI) partner reviewed the draft of the programme before setting it into a web system and suggested clarifying some points that might be difficult for general readers to understand. The second PPI partner enrolled monitors of a 1-hour semistructured interview, designed a recruitment poster, introduced the stakeholders (ie, policy-makers) to researchers, and advised on PR strategy for using SNS advertisement. Both women participated in an interview after using the web system, provided feedback, and took part in the recruitment of participants. Two PPI partners will participate in a discussion of the study findings and in setting up the implementation strategy after finishing RCT. The PPI process will be described based on the PPI handbook and reporting checklists.^{83 84}

Discussion

An internet-based, fully automated, eight-module self-help ACT programme ‘Happiness Mom’ designed for working women with preschool children was developed to improve their PWB. This study is the first RCT to examine the effectiveness of a fully automated iACT without face-to-face sessions for the PWB of a working population regardless of psychological symptoms. Some limitations of the study include the small size of the optional sessions of the programme (eg, parenting, relaxation skills). The intervention effectiveness will not be identified as the difference between iACT and those sessions. The study will examine the effectiveness of the intervention ‘Happiness Mom’ as a whole package. Second, this study will not assess multiple aspects of psychological flexibility using validated scales (eg, Acceptance and Action Questionnaire-II) to determine psychological processes connecting the intervention and the outcomes. We will not be able to identify what psychological process is most important if the intervention is found effective. Lastly, all the outcomes in this study will be assessed by the self-reporting questionnaire. In spite of these limitations, this study will contribute to the development of an internet-based self-care programme that is effective, feasible, low cost and accessible for improving the well-being of working mothers. This newly developed programme may provide psychological support for women to pursue their life careers according to their values. The web-based programme can be easily made available in other countries through translation in the future.

The authors appreciated Masao Tsuchiya and Satoshi Tajima, specialists in occupational psychology for contributing to the improvement of the iACT contents. The artist Ami Kito (amikitosketch), created the wonderful illustrations for Happiness Mom. Junko Tashiro and Toyoko Adachi, close friends of the first author and mothers of preschoolers, helped with developing the program and recruiting the participants. We also extend great thanks to our research assistant Keiko Miyake for building a web site and leaning system. The authors wish to thank the members of the Department of Psychiatric Nursing and Mental Health at The University of Tokyo for their advice in developing program.

Footnote

Contributors NK was in charge of this study, of supervising the process and provided his expert opinion. The authors (NS, KI and DN) organised the study design and developed the contents of the intervention. The authors (YS, YK and KT) developed the program. KW led the design of the statistical analyses. NS wrote the first draft of the protocol manuscript. All of the other authors revised the manuscript. All authors approved the final version of this manuscript.

Funding This work was supported by the Public Health Research Foundation (2020-2021 to NS) and the Occupational Health Promotion Foundation (2020-2021 to NS).

Competing interests None declared.

Patient consent for publication Obtained.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Ortiz-Ospina E, Tzvetkova S. Working Women: Key Facts and Trends in Female Labor Force Participation, 2017 Our World in Data. Available: https://ourworldindata.org/female-labor-force-participation-key-facts

² Ohu EA, Spitzmueller C, Zhang J, et al. When work-family conflict hits home: parental work-family conflict and child health. J Occup Health Psychol 2019; 24: 590–601. doi:10.1037/ocp0000145 http://www.ncbi.nlm.nih.gov/pubmed/30589277

³ The United Nations. Sustainable development goal 3: ensure healthy lives and promote well-being for all at all ages, 2017. Available: https://sustainabledevelopment.un.org/sdg3

⁴ The United Nations 2017 Sustainable Development Goal 5: Achieve gender equality and empower all women and girls

⁵ The United Nations,. Sustainable development goal 8: promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all 2017.

⁶ Grice MM, Feda D, McGovern P, et al. Giving birth and returning to work: the impact of work-family conflict on women"s health after childbirth. Ann Epidemiol 2007; 17: 791–8. doi:10.1016/j.annepidem.2007.05.002 http://www.ncbi.nlm.nih.gov/pubmed/17719242

⁷ Grice MM, McGovern PM, Alexander BH, et al. Balancing work and family after childbirth: a longitudinal analysis. Womens Health Issues 2011; 21: 19–27. doi:10.1016/j.whi.2010.08.003 http://www.ncbi.nlm.nih.gov/pubmed/21185988

⁸ Westman M, Etzion D, Gortler E. The Work-Family interface and burnout. Int J Stress Manag 2004; 11: 413–28. doi:10.1037/1072-5245.11.4.413

⁹ Farhadi A et al. The relationship between Work-family conflict, burnout dimensions and intention to leave among female nurses. Iran Journal of Nursing 2013; 26: 34–43.

¹⁰ Wang Y, Liu L, Wang J, et al. Work-family conflict and burnout among Chinese doctors: the mediating role of psychological capital. J Occup Health 2012; 54: 232–40. doi:10.1539/joh.11-0243-OA http://www.ncbi.nlm.nih.gov/pubmed/22790526

¹¹ Grice MM, McGovern PM, Alexander BH. Flexible work arrangements and work-family conflict after childbirth. Occup Med 2008; 58: 468–74. doi:10.1093/occmed/kqn090 http://www.ncbi.nlm.nih.gov/pubmed/18667390

¹² Halpern DF. Psychology at the intersection of work and family: recommendations for employers, working families, and policymakers. Am Psychol 2005; 60: 397–409. doi:10.1037/0003-066X.60.5.397 http://www.ncbi.nlm.nih.gov/pubmed/16045393

¹³ National Institute of Population and Social Security Researc. Annual population and social security surveys (the National fertility survey), 2015. Available: http://www.ipss.go.jp/ps-doukou/j/doukou15/doukou15_gaiyo.asp

¹⁴ Cooklin AR, Canterford L, Strazdins L, et al. Employment conditions and maternal postpartum mental health: results from the longitudinal study of Australian children. Arch Womens Ment Health 2011; 14: 217–25. doi:10.1007/s00737-010-0196-9 http://www.ncbi.nlm.nih.gov/pubmed/21116667

¹⁵ Cooklin AR, Westrupp E, Strazdins L, et al. Mothers" work-family conflict and enrichment: associations with parenting quality and couple relationship. Child Care Health Dev 2015; 41: 266–77. doi:10.1111/cch.12137 http://www.ncbi.nlm.nih.gov/pubmed/24673505

¹⁶ Killien MG, Habermann B, Jarrett M. Influence of employment characteristics on postpartum mothers" health. Women Health 2001; 33: 63–81. doi:10.1300/J013v33n01_05 http://www.ncbi.nlm.nih.gov/pubmed/11523641

¹⁷ Gjerdingen D, McGovern P, Attanasio L, et al. Maternal depressive symptoms, employment, and social support. J Am Board Fam Med 2014; 27: 87–96. doi:10.3122/jabfm.2014.01.130126 http://www.ncbi.nlm.nih.gov/pubmed/24390890

¹⁸ Mayberry LJ, Horowitz JA, Declercq E. Depression symptom prevalence and demographic risk factors among U.S. women during the first 2 years postpartum. J Obstet Gynecol Neonatal Nurs 2007; 36: 542–9. doi:10.1111/j.1552-6909.2007.00191.x http://www.ncbi.nlm.nih.gov/pubmed/17973697

¹⁹ Miyake Y, Tanaka K, Sasaki S, et al. Employment, income, and education and risk of postpartum depression: the Osaka maternal and child health study. J Affect Disord 2011; 130: 133–7. doi:10.1016/j.jad.2010.10.024 http://www.ncbi.nlm.nih.gov/pubmed/21055825

²⁰ Blanch A, Aluja A. Social support (family and supervisor), work–family conflict, and burnout: sex differences. Human Relations 2012; 65: 811–33. doi:10.1177/0018726712440471

²¹ Guille C, Frank E, Zhao Z, et al. Work-family conflict and the sex difference in depression among training physicians. JAMA Intern Med 2017; 177: 1766–72. doi:10.1001/jamainternmed.2017.5138 http://www.ncbi.nlm.nih.gov/pubmed/29084311

²² Frone MR. Work–family conflict and employee psychiatric disorders: the National comorbidity survey. J Appl Psychol 2000; 85: 888–95. doi:10.1037/0021-9010.85.6.888

²³ Marks NF. Does it hurt to care? caregiving, work-family conflict, and midlife well-being. J Marriage Fam 1998; 60: 951–66. doi:10.2307/353637

²⁴ Marks NF BL, Jun H. Family Roles and Well-Being During the Middle Life Course. In: Brim OG, Kessler RC, eds. How healthy are we? a national study of wellbeing at midlife. Chicago: University of Chicago Press, 2004: 514–49.

²⁵ World Health Organization. Promoting mental health: concepts, emerging evidence, practice. Geneva; 2005.

²⁶ Ryff CD. Beyond Ponce de leon and life satisfaction: new directions in quest of successful ageing. Int J Behav Dev 1989; 12: 35–55. doi:10.1177/016502548901200102

²⁷ Ryff CD. Psychological well-being revisited: advances in the science and practice of eudaimonia. Psychother Psychosom 2014; 83: 10–28. doi:10.1159/000353263 http://www.ncbi.nlm.nih.gov/pubmed/24281296

²⁸ Steptoe A, Deaton A, Stone AA. Subjective wellbeing, health, and ageing. The Lancet 2015; 385: 640–8. doi:10.1016/S0140-6736(13)61489-0

²⁹ Kim ES, Sun JK, Park N, et al. Purpose in life and reduced incidence of stroke in older adults: "The Health and Retirement Study". J Psychosom Res 2013; 74: 427–32. doi:10.1016/j.jpsychores.2013.01.013 http://www.ncbi.nlm.nih.gov/pubmed/23597331

³⁰ Kim ES, Sun JK, Park N, et al. Purpose in life and reduced risk of myocardial infarction among older U.S. adults with coronary heart disease: a two-year follow-up. J Behav Med 2013; 36: 124–33. doi:10.1007/s10865-012-9406-4 http://www.ncbi.nlm.nih.gov/pubmed/22359156

³¹ Boylan JM, Ryff CD. Psychological well-being and metabolic syndrome: findings from the midlife in the United States national sample. Psychosom Med 2015; 77: 548–58. doi:10.1097/PSY.0000000000000192 http://www.ncbi.nlm.nih.gov/pubmed/25984827

³² Mangelli L, Gribbin N, Büchi S, et al. Psychological well-being in rheumatoid arthritis: relationship to "disease" variables and affective disturbance. Psychother Psychosom 2002; 71: 112–6. doi:10.1159/000049354 http://www.ncbi.nlm.nih.gov/pubmed/11844948

³³ Wood AM, Joseph S. The absence of positive psychological (eudemonic) well-being as a risk factor for depression: a ten year cohort study. J Affect Disord 2010; 122: 213–7. doi:10.1016/j.jad.2009.06.032 http://www.ncbi.nlm.nih.gov/pubmed/19706357

³⁴ Hamilton NA, Gallagher MW, Preacher KJ, et al. Insomnia and well-being. J Consult Clin Psychol 2007; 75: 939–46. doi:10.1037/0022-006X.75.6.939 http://www.ncbi.nlm.nih.gov/pubmed/18085910

³⁵ Boyle PA, Buchman AS, Barnes LL, et al. Effect of a purpose in life on risk of incident Alzheimer disease and mild cognitive impairment in community-dwelling older persons. Arch Gen Psychiatry 2010; 67: 304–10. doi:10.1001/archgenpsychiatry.2009.208 http://www.ncbi.nlm.nih.gov/pubmed/20194831

³⁶ Fava GA, Rafanelli C, Cazzaro M, et al. Well-Being therapy. A novel psychotherapeutic approach for residual symptoms of affective disorders. Psychol Med 1998; 28: 475–80. doi:10.1017/S0033291797006363 http://www.ncbi.nlm.nih.gov/pubmed/9572104

³⁷ Hofmann SG, Sawyer AT, Fang A. The empirical status of the "new wave" of cognitive behavioral therapy. Psychiatr Clin North Am 2010; 33: 701–10. doi:10.1016/j.psc.2010.04.006 http://www.ncbi.nlm.nih.gov/pubmed/20599141

³⁸ Sin NL, Lyubomirsky S. Enhancing well-being and alleviating depressive symptoms with positive psychology interventions: a practice-friendly meta-analysis. J Clin Psychol 2009; 65: 467–87. doi:10.1002/jclp.20593 http://www.ncbi.nlm.nih.gov/pubmed/19301241

³⁹ Weiss LA, Westerhof GJ, Bohlmeijer ET. Can we increase psychological well-being? the effects of interventions on psychological well-being: a meta-analysis of randomized controlled trials. PLoS One 2016; 11: e0158092. doi:10.1371/journal.pone.0158092 http://www.ncbi.nlm.nih.gov/pubmed/27328124

⁴⁰ Hayes SC, Strosahl K, amp;Wilson KG. Acceptance and commitment therapy: an experiential approach to behavior change. New York: Guilford Press, 1999.

⁴¹ Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: the process and practice of mindful change. New York: Guilford Press, 2011.

⁴² Hayes SC, Luoma JB, Bond FW, et al. Acceptance and commitment therapy: model, processes and outcomes. Behav Res Ther 2006; 44: 1–25. doi:10.1016/j.brat.2005.06.006 http://www.ncbi.nlm.nih.gov/pubmed/16300724

⁴³ Blackledge JT, Hayes SC. Using acceptance and commitment training in the support of parents of children diagnosed with autism. Child Fam Behav Ther 2006; 28: 1–18. doi:10.1300/J019v28n01_01

⁴⁴ Brown FL, Whittingham K, Boyd RN, et al. Improving child and parenting outcomes following paediatric acquired brain injury: a randomised controlled trial of stepping stones triple P plus acceptance and commitment therapy. J Child Psychol Psychiatry 2014; 55: 1172–83. doi:10.1111/jcpp.12227 http://www.ncbi.nlm.nih.gov/pubmed/24635872

⁴⁵ Brown FL, Whittingham K, Boyd RN, et al. Does stepping stones triple P plus acceptance and commitment therapy improve parent, couple, and family adjustment following paediatric acquired brain injury? a randomised controlled trial. Behav Res Ther 2015; 73: 58–66. doi:10.1016/j.brat.2015.07.001 http://www.ncbi.nlm.nih.gov/pubmed/26255172

⁴⁶ Brown FL, Whittingham K, Sofronoff K. Parental experiential avoidance as a potential mechanism of change in a parenting intervention for parents of children with pediatric acquired brain injury. J Pediatr Psychol 2015; 40: 464–74. doi:10.1093/jpepsy/jsu109 http://www.ncbi.nlm.nih.gov/pubmed/25535318

⁴⁷ Coyne L, Murrell A. The joy of parenting: an acceptance and commitment therapy guide to effective parenting in the early years. New Harbinger Publications, 2009.

⁴⁸ Lunsky Y, Fung K, Lake J, et al. Evaluation of acceptance and commitment therapy (ACT) for mothers of children and youth with autism SpectrumDisorder. Mindfulness 2018; 9: 1110–6. doi:10.1007/s12671-017-0846-3

⁴⁹ O"Brien KM. Evaluating the effectiveness of a parent training protocol based on an acceptance and commitment therapy philosophy of parenting University of North Texas; 2011.

⁵⁰ Guendouzi J. "The Guilt Thing": Balancing Domestic and Professional Roles. J Marriage Fam 2006; 68: 901–9. doi:10.1111/j.1741-3737.2006.00303.x

⁵¹ Carolan S, Harris PR, Cavanagh K. Improving Employee Well-Being and Effectiveness: Systematic Review and Meta-Analysis of Web-Based Psychological Interventions Delivered in the Workplace. J Med Internet Res 2017; 19: e271. doi:10.2196/jmir.7583 http://www.ncbi.nlm.nih.gov/pubmed/28747293

⁵² Stratton E, Lampit A, Choi I, et al. Effectiveness of eHealth interventions for reducing mental health conditions in employees: a systematic review and meta-analysis. PLoS One 2017; 12: e0189904. doi:10.1371/journal.pone.0189904 http://www.ncbi.nlm.nih.gov/pubmed/29267334

⁵³ Barak A, Hen L, Boniel-Nissim M, et al. A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions. J Technol Hum Serv 2008; 26: 109–60. doi:10.1080/15228830802094429

⁵⁴ Cuijpers P, Donker T, van Straten A, et al. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010; 40: 1943–57. doi:10.1017/S0033291710000772 http://www.ncbi.nlm.nih.gov/pubmed/20406528

⁵⁵ Mailey EL, Huberty J, Irwin BC. Feasibility and effectiveness of a web-based physical activity intervention for working mothers. J Phys Act Health 2016; 13: 822–9. doi:10.1123/jpah.2015-0643 http://www.ncbi.nlm.nih.gov/pubmed/26999823

⁵⁶ Brown M, Glendenning A, Hoon AE, et al. Effectiveness of Web-Delivered acceptance and commitment therapy in relation to mental health and well-being: a systematic review and meta-analysis. J Med Internet Res 2016; 18: e221. doi:10.2196/jmir.6200 http://www.ncbi.nlm.nih.gov/pubmed/27558740

⁵⁷ Kelson J, Rollin A, Ridout B, et al. Internet-Delivered acceptance and commitment therapy for anxiety treatment: systematic review. J Med Internet Res 2019; 21: e12530. doi:10.2196/12530 http://www.ncbi.nlm.nih.gov/pubmed/30694201

⁵⁸ O"Connor M, Munnelly A, Whelan R, et al. The efficacy and acceptability of Third-Wave behavioral and cognitive eHealth treatments: a systematic review and meta-analysis of randomized controlled trials. Behav Ther 2018; 49: 459–75. doi:10.1016/j.beth.2017.07.007 http://www.ncbi.nlm.nih.gov/pubmed/29704973

⁵⁹ Lappalainen P, Kaipainen K, Lappalainen R, et al. Feasibility of a personal health technology-based psychological intervention for men with stress and mood problems: randomized controlled pilot trial. JMIR Res Protoc 2013; 2: e1. doi:10.2196/resprot.2389 http://www.ncbi.nlm.nih.gov/pubmed/23611946

⁶⁰ Puolakanaho A, Tolvanen A, Kinnunen SM, et al. A psychological flexibility -based intervention for Burnout:A randomized controlled trial. J Contextual Behav Sci 2020; 15: 52–67. doi:10.1016/j.jcbs.2019.11.007

⁶¹ Kinnunen SM, Puolakanaho A, Tolvanen A, et al. Improvements in mindfulness facets mediate the alleviation of burnout dimensions. Mindfulness 2020; 11: 2779–92. doi:10.1007/s12671-020-01490-8

⁶² Kinnunen SM, Puolakanaho A, Tolvanen A, et al. Does mindfulness-, acceptance-, and value-based intervention alleviate burnout?—A person-centered approach. Int J Stress Manag 2019; 26: 89–101. doi:10.1037/str0000095

⁶³ Kinnunen SM, Puolakanaho A, Mäkikangas A, et al. Does a mindfulness-, acceptance-, and value-based intervention for burnout have long-term effects on different levels of subjective well-being? Int J Stress Manag 2020; 27: 82–7. doi:10.1037/str0000132

⁶⁴ Burton NW, Pakenham KI, Brown WJ. Feasibility and effectiveness of psychosocial resilience training: a pilot study of the ready program. Psychol Health Med 2010; 15: 266–77. doi:10.1080/13548501003758710 http://www.ncbi.nlm.nih.gov/pubmed/20480432

⁶⁵ Schotanus-Dijkstra M, Drossaert CHC, Pieterse ME, et al. An early intervention to promote well-being and flourishing and reduce anxiety and depression: a randomized controlled trial. Internet Interv 2017; 9: 15–24. doi:10.1016/j.invent.2017.04.002 http://www.ncbi.nlm.nih.gov/pubmed/30135833

⁶⁶ Chan A-W, Tetzlaff JM, Altman DG, et al. Spirit 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 2013; 158: 200–7. doi:10.7326/0003-4819-158-3-201302050-00583 http://www.ncbi.nlm.nih.gov/pubmed/23295957

⁶⁷ Hayes SC. Get out of your mind and into your life: the new acceptance and commitment therapy. New Harbinger Publications, 2005.

⁶⁸ Bailey A, Ciarrochi J, Hayes L. Get out of your mind and into your life for teens: a guide to living an extraordinary life. New Harbinger Publications, 2012.

⁶⁹ Hayes SC, Levin ME, Plumb-Vilardaga J, et al. Acceptance and commitment therapy and contextual behavioral science: examining the progress of a distinctive model of behavioral and cognitive therapy. Behav Ther 2013; 44: 180–98. doi:10.1016/j.beth.2009.08.002 http://www.ncbi.nlm.nih.gov/pubmed/23611068

⁷⁰ Sanders MR. Triple P-Positive parenting program: towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clin Child Fam Psychol Rev 1999; 2: 71–90. doi:10.1023/A:1021843613840 http://www.ncbi.nlm.nih.gov/pubmed/11225933

⁷¹ Chawla N, Ostafin B. Experiential avoidance as a functional dimensional approach to psychopathology: an empirical review. J Clin Psychol 2007; 63):: 871–90. doi:10.1002/jclp.20400 http://www.ncbi.nlm.nih.gov/pubmed/17674402

⁷² Kabat-Zinn J. Wherever you go, there you are: mindfulness meditation in everyday life. Hachette Books, 2009.

⁷³ Neff K. Self-compassion: the proven power of being kind to yourself. Harper Collins, 2011.

⁷⁴ Neff KD, Rude SS, Kirkpatrick KL. An examination of self-compassion in relation to positive psychological functioning and personality traits. J Res Pers 2007; 41: 908–16. doi:10.1016/j.jrp.2006.08.002

⁷⁵ Shiota MN, Campos B, Oveis C, et al. Beyond Happiness: building a science of discrete positive emotions. Am Psychol 2017; 72: 617–43. doi:10.1037/a0040456 http://www.ncbi.nlm.nih.gov/pubmed/29016167

⁷⁶ Uvnäs-Moberg K, Handlin L, Petersson M. Self-soothing behaviors with particular reference to oxytocin release induced by non-noxious sensory stimulation. Front Psychol 2014; 5: 1529. doi:10.3389/fpsyg.2014.01529 http://www.ncbi.nlm.nih.gov/pubmed/25628581

⁷⁷ Sanders MR. Development, evaluation, and multinational dissemination of the triple P-Positive parenting program. Annu Rev Clin Psychol 2012; 8: 345–79. doi:10.1146/annurev-clinpsy-032511-143104 http://www.ncbi.nlm.nih.gov/pubmed/22149480

⁷⁸ Jerath R, Crawford MW, Barnes VA, et al. Self-Regulation of breathing as a primary treatment for anxiety. Appl Psychophysiol Biofeedback 2015; 40: 107–15. doi:10.1007/s10484-015-9279-8 http://www.ncbi.nlm.nih.gov/pubmed/25869930

⁷⁹ Carlson CR, Hoyle RH. Efficacy of abbreviated progressive muscle relaxation training: a quantitative review of behavioral medicine research. J Consult Clin Psychol 1993; 61: 1059–67. doi:10.1037/0022-006X.61.6.1059 http://www.ncbi.nlm.nih.gov/pubmed/8113484

⁸⁰ Ryff CD. Happiness is everything, or is it? explorations on the meaning of psychological well-being. J Pers Soc Psychol 1989; 57: 1069–81. doi:10.1037/0022-3514.57.6.1069

⁸¹ Sasaki N, Watanabe K, Imamura K, et al. Japanese version of the 42-item psychological well-being scale (PWBS-42): a validation study. BMC Psychol 2020; 8: 75. doi:10.1186/s40359-020-00441-1 http://www.ncbi.nlm.nih.gov/pubmed/32690082

⁸² Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 2017; 12: 108. doi:10.1186/s13012-017-0635-3 http://www.ncbi.nlm.nih.gov/pubmed/28851459

⁸³ Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ 2017; 358: j3453. doi:10.1136/bmj.j3453 http://www.ncbi.nlm.nih.gov/pubmed/28768629

⁸⁴ National Institute for Health Research. Patient and public involvement in health and social care research; 2018.