Supporting workforce practice change: protocol for a pilot study of a motivational interviewing virtual client software tool for health professionals

Abstract

Introduction

Motivating behavioural change during client consultations is of crucial importance across all health professions to address the growing burden of chronic conditions. Yet health professionals often lack the skills and confidence to use evidence-based counselling interventions to support clients’ behavioural change and mobilise clients’ resources and self-efficacy for change to address their long-term needs.

Aims

This pre–post pilot study will develop a motivational interviewing (MI) virtual client training tool for health professionals and test the effectiveness of the educational content and usability of the virtual client interaction.

Methods and analysis

Postgraduate students across a range of health disciplines will be recruited. Data assessing attitudes towards preventive healthcare will be collected using a modified version of the Preventive Medicine Attitudes and Activities Questionnaire. Conversations with the virtual client will be analysed using the Motivational Interviewing Treatment Integrity code to assess changes in MI skills. The System Usability Scale will be used to assess the usability of the virtual client training tool.

Ethics and dissemination

This protocol was approved by the Flinders University Social and Behavioural Research Ethics Committee in May 2019. The results of the pilot study will inform the development of an avatar-based mobile application consisting of MI teaching and interactions with a generic virtual client that can be easily adapted to multiple scenarios.

Correspondence to Dr Candice Oster; [email protected]

Strengths and limitations of this study

• The study will pilot online motivational interviewing (MI) education to provide health professionals with an accessible, cost-effective training resource for the development and maintenance of MI competence.

• Potential limitations of the design include self-selection bias and potentially low response rates given that postgraduate students are often time poor.

• This pilot study represents the initial step in the development of a mobile application providing comprehensive MI training and interaction with a virtual client so health professionals can develop, practise and receive feedback on their MI skills.

Introduction

Non-communicable, largely chronic conditions account for around three in five global deaths, attributed in particular to cardiovascular disease, cancer, chronic lung disease and diabetes.¹ Lifestyle risk factors (eg, diet, body mass index and smoking) are the leading risk factors for chronic conditions.² Modifiable health behaviours are key targets of health promotion and interventions to reduce the burden of chronic conditions.³ While it can be difficult to provide interventions for entire populations, primary prevention at the individual client level is recognised as an effective means of supporting behavioural change.³ Motivating behavioural change during client consultations is therefore of crucial importance across all health professions.^{4 5} Yet health professionals often lack the skills and confidence to use effective, evidence-based counselling interventions to support behavioural change.

Motivational interviewing (MI) is a counselling method that enhances clients’ motivation to change through resolving ambivalence about health behavioural change. Backed up by 40 years of research, MI is a key skill for health professionals internationally for supporting their clients to make and maintain positive behavioural changes.^6–9 Research has shown the effectiveness of MI across a range of health professional groups and settings.^{10 11} Yet it is a complex skill and health professionals have struggled to translate these skills into everyday practice.¹²

Health professionals seeking to improve their ability and confidence to counsel clients about changing behaviours need access to effective education to facilitate and sustain this practice change. The principal method of MI training is through face-to-face workshops, often including experiential exercises such as role-plays or use of a standardised patient.¹³ The current approach to MI training has tended to focus on ‘more general or introductory MI training’,^{13, p106} which fails to adequately develop health professionals’ MI skills, particularly over the long term. Miller and Moyers¹⁴ have suggested that in order to develop MI competence, MI training should ideally incorporate eight stages across two phases (phase 1: enhancing clients’ motivation for change; phase 2: strengthening and consolidating commitment to change). This staged approach aims to embed MI skills and support the maintenance of these skills in health professionals’ future practice.¹⁴ The application of this staged approach has been explored in a systematic review of training in MI, with the results demonstrating that most of the MI training studies focused primarily on a small number of stages in phase 1 of MI training.¹³

In addition to limitations in the extent to which MI skills are consistently embedded in health professionals’ long-term practice, MI workshops often fail to address the need for health professionals to develop and reinforce their MI skills in a non-threatening environment in which they can gain confidence, make mistakes safely and learn from them. Health professionals also need to revisit, practise, reflect and test how they operationalise the MI skills they have learnt to their everyday practice. Furthermore, workshops are less accessible to rural, remote and busy health professionals. It is well documented that distance from and access to education providers is a barrier to further education.^{15 16} Consequently, there is a move towards the development and provision of technology-delivered MI training, using virtual standardised patients, which aims to address these challenges.

For example, Kognito has developed a number of role play simulations using MI communication skills (open questions, affirmations, reflections and summaries; MI Spirit—partnership, acceptance, compassion and evocation), along with a range of communication strategies based on social/cognitive models and neuroscience.¹⁷ The simulations focus on specific behaviours and settings, such as a parent talking with a child about substance abuse, a veteran who is worried about a buddy, and a healthcare professional conducting screening and brief interventions with adolescents. Carpenter et al ¹⁸ developed a multimedia instructional tutorial to teach MI-based brief intervention strategies for smokers, using a Role-play Internet Simulation. Vicky is another example involving an MI-based brief intervention.¹⁹ Vicky is a virtual coach who teaches the Screening Brief Intervention and Referral to Treatment intervention for clients with alcohol or other drug use problems in the emergency room setting.

As these examples demonstrate, current approaches to technology-delivered MI training tend to focus on particular health behaviours and particular settings. Furthermore, there is a lack of MI specific training that teaches MI in depth across all of Miller and Moyers’¹⁴ eight stages. No ‘one stop’ technology-based MI training was identified that covers the range of client types, conditions and behaviours usually seen in real-world practice, given working with clients with diverse multimorbidity is the norm for most health professionals. A contributory factor may be the laborious nature of existing virtual client (VC) creation; for example, developing a smoking cessation case study does not make it easier to develop a weight loss scenario, as each case is comprised of the specific words used in the virtual interviews. Integration of advances in techniques of dialogue knowledge representation and management and ontology integration²⁰ can address this problem.

The pilot study discussed here will involve the development and pilot testing of technology-delivered MI training to inform future development of a VC-based MI learning system covering multiple conditions, behaviours and client groups to provide opportunities for health professionals to develop, practise and receive feedback on their MI skills. VCs are increasingly being used in teaching various skills to health professionals, and have been effective in increasing knowledge, confidence and skills.^{21 22} In the pilot study, we aim to develop and test a virtual client conversation tool (V-CCT) for use in teaching MI. The V-CCT will be integrated within an online MI course following Miller and Moyer’s¹⁴ eight stages of MI training. Development of the V-CCT and educational content began in January 2019 with participant recruitment beginning in August 2019. Data collection is due to be finalised by May 2020 with an estimated completion date for the pilot being July 2020.

Methods and analysis

A pre–post pilot study is currently underway at Flinders University, South Australia, testing the following aspects of technology-delivered MI training:

• The effectiveness of MI teaching delivered via e-learning following Miller and Moyer’s¹⁴ eight stages of MI training.

• The usability of a V-CCT.

Development of an online MI course

The MI course has been developed following Miller and Moyer’s¹⁴ eight stages of MI training (see table 1).

Table 1

The eight stages of MI training

Stage	Description
The Spirit of MI	Partnership, acceptance (absolute worth; accurate empathy, autonomy support, affirmation), compassion and evocation (evoking the client’s own motivation and resources for change)
OARS	Client-centred counselling skills (open questions, affirmations, reflections, summaries)
Recognising and reinforcing change talk	Identifying and reinforcing desire, ability, reason, need and commitment to change
Eliciting and strengthening change talk	Using OARS skills to elicit change talk and commitment language
Rolling with resistance	Importance of not directly refuting the client’s arguments against change (resisting the righting reflex) and strategies to roll with resistance
Developing a change plan	How to work with the client to develop a change plan
Consolidating client commitment	Eliciting and consolidating the client’s commitment to make the change
Switching between MI and other approaches	MI can be combined flexibly with other counselling methods

MI, motivational interviewing.

The course has been developed by MI training experts within Flinders University. The course involves a series of recorded lectures, links to YouTube videos demonstrating MI, downloadable resources and quizzes for checking and reinforcing knowledge. The course will be delivered using the university’s Learning Management System, a Moodle-based platform.

Development of the V-CCT

For the purpose of the pilot study, the V-CCT focuses on a conversation with a virtual client about diet and weight loss. The tool has been developed based on a case study of a VC, Anna, developed specifically for the study.

Villaume et al ²³ identify three classes of interactive VCs. In the first class are VCs that use a restricted set of questions that the user can click on, generating an automated response from the client. The sets of questions are independent and can, therefore, be asked in any order. The second class of VCs is scripted to simulate a natural flow of interaction between user and the VC, while still using a restricted set of questions. This involves the VC responding to the selected question followed by a list of further responses from which the user can select. The third class involves freeform spoken input by the user with the computer determining the exact wording of the VC’s response in light of what the user has spoken (see eg, Hubal et al ²⁴). In this class, VC responses are generally delivered in the form of videos or through the use of avatars. According to Sweller’s cognitive load theory,²⁵ presenting material using visual, auditory and/or gestural interfaces helps to reduce cognitive load, allowing us to either absorb more information or to engage at a higher cognitive level.

In this study, we have used a hybrid of classes two and three, with the VC response being in text form (with a view to developing a VC avatar in the next stage of the project, underpinned by the pilot study results). The user’s interaction with the VC involves typing in a question/response, and then selecting the closest match from a restricted set of responses (or if there is no close match, then selecting whichever response the user deems appropriate). The VC then responds (in text form in the pilot version) and the user again types in their response and selects the closest match from the restricted set of responses. The purpose of adopting this approach is to better facilitate learning of MI by the user actively practising what they would say (typing their response), while still mimicking a conversation flow.

The conversation flow options have been designed to incorporate four conversation flows; an MI-consistent flow, an MI-inconsistent flow and two neutral conversation flows. The neutral conversation flows are designed to be either somewhat MI consistent or somewhat MI inconsistent. This allows us to limit the number of VC responses while still maintaining a logical flow in the conversation. The conversation flows are interconnected (with one response option for each conversation flow presented in response to the VC) such that the user can move between MI-consistent, MI-inconsistent and neutral conversations. This is an instructional process supporting conscious effort by the user and reinforcing the acquisition of MI knowledge through activities that enhance higher order thinking (or deeper learning) rather than the replication of content.²⁶ The interconnection of the conversation flows maximises critical thinking and thinking mindfully, providing opportunities for learners to consider how they can actively use MI skills in engaging with the VC.^{25 27}

Development of the conversation flows

In order to develop the conversation flows we began with the MI-consistent flow, informed by the case study of Anna. The conversation follows the four processes of MI, namely engaging (establishing a working relationship), focusing (clarifying the client’s agenda), evoking (eliciting and reinforcing the client’s own arguments for behavioural change) and planning (guiding the client to develop and commit to a plan of action). This conversation was then used as the basis for developing the health professional text options and the VC responses in the MI-inconsistent and two neutral conversation flows, ensuring that the user could move between the four conversation pathways to enable them to get back on track when diverging from the MI-consistent interaction. A selection of the script flow chart is shown in figure 1. HP refers to health professional, C is the client, the numbers are the set of questions/responses from which the user selects, and the letters refer to the MI-inconsistent (A), neutral (B and C) and MI-consistent (D) conversation flows. While the user always gets to choose from one of the four response options (A, B, C or D), these are presented in the V-CCT in random order to mitigate against the user guessing which option to select.

Figure 1. Script flow chart.

Interaction with the VC

The V-CCT is accessed via a website where the user is introduced to Anna in a short video (see screenshot in figure 2). They are then asked to enter either the precourse dialogue (to be completed prior to completing the MI course) or the postcourse dialogue (to be completed after completing the course). The dialogue screen (showing user inputted text and text selection options) can be seen in figure 3.

Figure 2. Introduction to the virtual client.

Figure 3. Dialogue screen.

The dialogue, derived from the script flow chart (figure 1), was modelled inside of a Standard Query Language database. This database is then queried each time the user makes a response, generating both a response from Anna that is displayed to the user and a list of four multiple-choice options that the user will select from in the next round. While the user is interacting with Anna, each iteration of the conversation is recorded, linking what the user typed in the free text section, their multiple-choice selection and the response from Anna. After the conversation is over, the user’s ‘attempt’ is logged with a date and time for possible retrospective analysis to ensure that the user has not done both conversations straight after each other (ie, without completing the MI course in between).

After progressing through the conversation with Anna, the user then receives feedback about how they went, focusing on the extent to which they selected MI-consistent, MI-inconsistent and neutral text. No feedback is provided during the conversation flow in the pilot version of the V-CCT, nor will users be shown which are the MI-consistent responses, because the V-CCT will be used as part of the pre–post pilot testing of the effectiveness of the MI course content. We believe that provision of this type of formative feedback during the pilot testing would prevent an objective assessment of MI skill development; however, formative feedback will be provided in later versions of the V-CCT.

Recruitment

Participants for the pilot study will be recruited via Flinders University postgraduate courses for students already working within or training to enter into a range of health professions (eg, medicine, nursing, psychology, physiotherapy, nutrition and dietetics, occupational therapy). This will provide a potential participant pool of around 200 students. While interaction with the VC begins with a discussion of blood glucose levels, users are informed that the conversation will be focusing on behavioural change relating to diet and exercise, which has application to all of the health professional student groups, rather than medical management.

Participants will be invited to take part in the online MI course via an announcement through the Flinders University Learning Management System. Consent to participate will be obtained by students signing a consent form and emailing it to administrative staff not involved in the study, who will enrol students in the online course. Participants will interact with the V-CCT prior to going through the online MI educational content and again at the end. Access to the VC will be through a link to a website.

Data collection

In order to test the effectiveness of the online MI course delivered using the eight stages of teaching MI, students will be asked to complete the Preventive Medicine Attitudes and Activities Questionnaire (Modified) (PMAAQ) prior to undertaking the course. They will be asked to complete the PMAAQ again at the end of the course. The PMAAQ in its original form was developed by Yeazel et al ²⁸ and is an 85-item tool that addresses physicians’ health prevention behaviours and preventive healthcare attitudes. The questionnaire has its theoretical underpinnings in social cognitive theory with the key constructs being ‘self-efficacy (confidence in performing a behaviour), situation (one’s perception of the environment), behavioural capacity (knowledge and skill to perform a behaviour), expectancies (values placed on a given outcome) and expectations (anticipated outcomes of a behaviour)’(28, p.87). The tool was initially assessed for validity and reliability in the form of cardiovascular disease subscales²⁹ and the complete instrument assessed for validity, reliability and stability results.²⁸ The PMAAQ has been used in a number of studies, both in its original form and with various modifications for particular settings and research questions.^30–32 The modified version we are using was used in previous research to assess the motivation of health providers who were trained to provide SagePlus lifestyle counselling to use MI in their practice.³³ This version has been further modified to exclude four questions based on the results of reliability testing we conducted of the questionnaire, leaving nine items relating to preventive healthcare attitudes (see box 1) (the reliability of this modification will be assessed in the pilot study). Responses are on a 5-point Likert scale from strongly agree to strongly disagree.

Box 1

Modified Preventive Medicine Attitudes and Activities Questionnaire questions

Question

Behavioural change counselling is an effective use of my time as a healthcare provider.

For most clients, health education does little to promote their adherence to a healthy lifestyle.

I am less effective than professional counsellors in getting clients to change their behaviour.

Clients without symptoms will rarely change their behaviour on the basis of my advice.

Most clients try to change their lifestyle if I advise them to do so.

It is difficult to understand why clients can’t meet the goals they have set with me.

It would be difficult to change my routine of lifestyle counselling to include motivational interviewing.

Clients prefer being told what to do over helping to come up with a plan themselves.

Doing lifestyle counselling using motivational interviewing takes longer than traditional methods.

The effectiveness of the online MI course will be further evaluated by analysing the users’ inputted text from the V-CCT (precourse vs postcourse) using the Motivational Interviewing Treatment Integrity (MITI) code.³⁴ The MITI is a reliable and valid MI fidelity tool that is widely used to assess the degree to which an MI conversation reflects the paradigms of MI.^{35 36} The MITI is particularly useful in this pilot study because it assesses the health professional’s language (ie, the text the user inputs into the V-CCT) and not the client’s language (which has been predetermined in the V-CCT). The MITI incorporates four global scales (Cultivating Change Talk, Softening Sustain Talk, Partnership and Empathy), rated on a 5-point Likert scale measuring how well or poorly the clinician meets the description of the dimension being measured. Global scores are summarised as Technical Global (the score of Cultivating Change Talk plus Softening Sustain Talk divided by two) and Relational Global (the score of Partnership plus Empathy divided by two). There are also ten behavioural counts, which are summarised as pecentage complex reflections, reflection-to-question ratio, total MI-adherent behaviours and total MI non-adherent behaviours. Proficiency and competency thresholds are provided for each dimension.

In order to test the usability of the V-CCT, the post-MI course survey also includes questions exploring users’ views of the V-CCT using the System Usability Scale (SUS). The SUS is considered to be ‘the most widely used standardised questionnaire for the assessment of perceived usability’.^{31, p.1} The SUS includes 10 questions assessed on a 5-point Likert scale from strongly agree to strongly disagree (see box 2). The scale has been assessed for validity and reliability³⁷ and used in previous research with VCs.³⁸

Box 2

System Usability Scale questions

I think that I would like to use this conversation tool frequently.

I found this tool unnecessarily complex.

I thought the tool was easy to use.

I think that I would need the support of a technical person to use this tool.

I found the various functions in the tool were well integrated.

I thought there was too much inconsistency in this tool.

I would imagine that most people would learn to use this tool very quickly.

I found the tool very awkward to use.

I felt very confident using the tool.

I needed to learn a lot of things before I could get going with the tool.

Finally, the postcourse questionnaire includes additional closed and open-ended questions about the course itself, as described in table 2.

Table 2

Questions exploring participants’ views of the MI course

Question	Response options*
How likely are you to recommend this course to a friend or classmate?	10-point Likert scale from not at all likely to extremely likely
How challenging was this course?	5-point Likert scale from extremely challenging to not challenging at all
How much did you learn from this course?	5-point Likert scale from a great deal to nothing at all
How reasonable or unreasonable was the workload for this course?	7-point Likert scale from extremely reasonable to extremely unreasonable
How well did this course meet your expectations?	5-point Likert scale from extremely well to not well at all
What did you like most about this course?	Free-text response
What did you like least about this course?	Free-text response
How could this course be improved?	Free-text response

*Questions for this section were amalgamated from a range of educational surveys, hence the variation in response options.

MI, motivational interviewing.

Demographic data will also be collected, as described in table 3.

Table 3

Demographic questions

Question	Response options
What is your age?	18–24; 25–34; 35–44; 45–54; 55–64; 65 or older
What is your gender?	Male; female; other
Have you had motivational interviewing training through another source this semester (eg, through your postgraduate courses) in addition to participating in this course?	Yes; no
Are you currently, or have you previously, worked as a health professional?	Yes; no
If yes, in what role?	GP; RN; dietitian; physiotherapist; occupational therapist; social worker; psychologist; other (please specify)
About how many years have you been, or did you, work in that role?	Less than 1 year; At least 1 but less than 3 years; at least 3 years but less than 5 years; at least 5 but less than 10 years; 10 years or more
What is your main client group?	Free-text response
Which of the following best describes the geographical location in which you work?	Metropolitan; regional; rural; remote
Do you currently, or have you previously, used motivational interviewing in your consultations with clients?	Yes; No

GP, general practitioner; RN, registered nurse.

Analysis

This study will pilot the MI course, the V-CCT and logistical aspects of the study for future research.³⁹ This includes using the results of this study to ‘obtain empirical estimates of statistical parameters to inform power calculations, and other design effects, of a subsequent trial’.^{39, p.3} Negatively worded items on the PMAAQ will be reverse scored and mean scores calculated for each item. Two members of the research team will code the user inputted text using the MITI and the scores will be averaged. Mean scores will be calculated for each summary domain (global technical, global relational, percentage complex reflection, reflection-to-question ratio, MI-adherent behaviours and MI non-adherent behaviours). Scoring of the SUS will follow Lewis.³⁷ Blank items will be given a raw score of 3. For odd numbered items, 1 will be subtracted from the raw score, with the raw score subtracted from 5 for even-numbered items. The sum of adjusted scores will be computed, which will be multiplied by 2.5 to get a standard SUS score. Mean scores on the questions exploring participants’ views of the MI course will be calculated.

Relevant to the pilot design of this project, statistical analyses will mostly be descriptive and specific techniques will be selected once the nature of the data is known, for example, distribution of outcomes. Analysis of precourse and postcourse questionnaires and MITI scores will be through t-tests or analyses of variances. Cronbach’s alpha’s will be calculated for the PMAAQ (assuming sufficient sample size). Alpha error will be set to p<0.05. Answers to open-ended questions will be analysed using content analysis.⁴⁰

Patient and public involvement

The research question is informed by the needs of patients to have health professionals educated to support them to self-manage their chronic conditions. No patients were involved in the design of the study or recruitment to and conduct of the study.

Ethics and dissemination

Ethical risk to participants is likely to be minimal. Participants’ university lecturers will not be aware of who has taken part in the study. Pre–post data will be linked using a unique identifier to maintain anonymity, with the spreadsheet of identifiers maintained by administrative personnel with no involvement of the researchers.

The results of the pilot study will be used to inform the development of an avatar-based mobile application consisting of MI teaching using the eight stages identified by Miller and Moyers¹⁴ and interactions with a generic VC that can be easily adapted to multiple scenarios.

Discussion

Behavioural change support for chronic condition management is fast becoming one of the most significant challenges for health professionals worldwide. MI can be integrated into health professionals’ consultations to address clients’ ambivalence to health behavioural change as it arises. It can also be used to support retention and adherence with therapies.⁴¹ This can facilitate early intervention to address health behavioural factors, rather than waiting for clients to reach a crisis stage in their health.^{42 43}

Health professionals seeking to improve their ability and confidence to counsel and support their clients to change behaviours need access to effective education to support practice change. Development of the MI V-CCT will inform the development of a comprehensive MI training platform to provide an accessible, cost-effective, cross-disciplinary resource that supports the development and maintenance of MI competence beyond basic training levels. While the pilot work is situated in a single scenario (diet and weight loss), results of the pilot study will be used to establish a platform from which to undertake the development of a generic virtual MI client that can be easily adapted to multiple scenarios.

A promising approach to achieving this would be to incorporate abstract rules or ‘recipes’ about the MI process into the software.²⁰ Such a recipe encapsulates an abstract generalisation about the dynamics of an MI session, which holds regardless of the health behaviour that is in focus. In addition, other, lower-level rules define the explicit content of these summary statements for a particular health behaviour scenario. In combination with software that enacts the higher-level recipe, developing a subsequent scenario would involve only modifying portions of the lower-level rules, while the more abstract high-level rules are reused as they are. Bickmore et al ²⁰ show that this approach allowed a second scenario to be developed in only 4% of the person hours that had been required to develop a first scenario, while reusing 22% of the software code. Such a software system will constitute a significant advance in the methodology of virtual client development and will greatly accelerate the ability of MI trainers to create new training material and adapt existing lessons.

Allowing students to type in what they would have said next (their most-preferred ‘dialogue move’) in advance of seeing the multiple choices available, also provides us with valuable data for enriching the existing scenario as well as informing the development of subsequent scenarios. MI-adherent student dialogue moves may be used as a source of alternative verbal formulations that can be added in order to make the system more interesting to use, but we envisage that the main value of student moves will be in suggesting interesting common mistakes that students make, in order to incorporate these into MI non-adherent paths in the dialogue.

Building on the pilot study in this way will ensure the tool is relevant to a range of health professionals and the varied client groups with which they interact. Such technology-delivered training is particularly valuable for health professionals in rural and remote locations where it is difficult to provide the same levels of face-to-face support for training as is available to those in urban settings. Additionally, technology-delivered training provides opportunities for ongoing reinforcement and consolidation of skill development and maintenance that is not generally provided with face-to face delivery of training.

Footnote

Contributors CO, AS and SL designed the study. All authors provided methodological input into the study design. CO, JL, SL and AS were involved in designing the virtual client conversation. CA, RL, DP and AM were involved in designing and developing the virtual client interaction. CO, AM and BL developed the content for the online learning. All authors were involved in drafting the manuscript or revising it critically for important content. All authors have read, reviewed and given final approval of the version to be published.

Funding This work was supported by Flinders Foundation Health Seed grant number 3000010982.

Competing interests None declared.

Patient consent for publication Not required.

Ethics approval Ethical approval for the pilot study was granted by the Flinders University Social and Behavioural Research Ethics Committee (project number 8338) in May 2019.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Hajat C, Stein E. The global burden of multiple chronic conditions: a narrative review. Prev Med Rep 2018; 12: 284–93. doi:10.1016/j.pmedr.2018.10.008

² Lim SS, Vos T, Flaxman AD, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the global burden of disease study 2010. The Lancet 2012; 380: 2224–60. doi:10.1016/S0140-6736(12)61766-8

³ Araújo-Soares V, Hankonen N, Presseau J, et al. Developing behavior change interventions for self-management in chronic illness. Eur Psychol 2019; 24: 7–25. doi:10.1027/1016-9040/a000330

⁴ Sibille K, Greene A, Bush JP. Preparing physicians for the 21st century: targeting communication skills and the promotion of health behavior change. Ann Behav Sci Med Educ 2010; 16: 7–13. doi:10.1007/BF03355111

⁵ Vallis M, Lee-Baggley D, Sampalli T, et al. Equipping providers with principles, knowledge and skills to successfully integrate behaviour change counselling into practice: a primary healthcare framework. Public Health 2018; 154: 70–8. doi:10.1016/j.puhe.2017.10.022

⁶ Rubak S, Sandbaek A, Lauritzen T, et al. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract 2005; 55: 305–12.

⁷ Lai DT, Cahill K, Qin Y, et al. Motivational interviewing for smoking cessation. Cochrane Database Syst Rev 2010: CD006936. doi:10.1002/14651858.CD006936.pub2

⁸ Lundahl BW, Kunz C, Brownell C, et al. A meta-analysis of motivational interviewing: twenty-five years of empirical studies. Res Soc Work Pract 2010; 20: 137–60. doi:10.1177/1049731509347850

⁹ VanBuskirk KA, Wetherell JL. Motivational interviewing with primary care populations: a systematic review and meta-analysis. J Behav Med 2014; 37: 768–80. doi:10.1007/s10865-013-9527-4

¹⁰ Brug J, Spikmans F, Aartsen C, et al. Training dietitians in basic motivational interviewing skills results in changes in their counseling style and in lower saturated fat intakes in their patients. J Nutr Educ Behav 2007; 39: 8–12. doi:10.1016/j.jneb.2006.08.010

¹¹ Brobeck E, Bergh H, Odencrants S, et al. Primary healthcare nurses’ experiences with motivational interviewing in health promotion practice. J Clin Nurs 2011; 20: 3322–30. doi:10.1111/j.1365-2702.2011.03874.x

¹² Walters ST, Matson SA, Baer JS, et al. Effectiveness of workshop training for psychosocial addiction treatments: a systematic review. J Subst Abuse Treat 2005; 29: 283–93. doi:10.1016/j.jsat.2005.08.006

¹³ Madson MB, Loignon AC, Lane C. Training in motivational interviewing: a systematic review. J Subst Abuse Treat 2009; 36: 101–9. doi:10.1016/j.jsat.2008.05.005

¹⁴ Miller WR, Moyers TB. Eight stages in learning motivational interviewing. Am J Teach Addict 2006; 5: 3–17. doi:10.1300/J188v05n01_02

¹⁵ Meadley L, Conway J, McMillan M. Education and training needs of nurses in general practice. Aust J Prim Health 2004; 10: 21–7. doi:10.1071/PY04004

¹⁶ Eley R, Fallon T, Soar J, et al. The status of training and education in information and computer technology of Australian nurses: a national survey. J Clin Nurs 2008; 17: 2758–67. doi:10.1111/j.1365-2702.2008.02285.x

¹⁷ Albright G, Bryan C, Adam C, et al. Using virtual patient simulations to prepare primary health care professionals to conduct substance use and mental health screening and brief intervention. J Am Psychiatr Nurses Assoc 2018; 24: 247–59. doi:10.1177/1078390317719321

¹⁸ Carpenter KM, Watson JM, Raffety B, et al. Teaching brief interventions for smoking cessation via an interactive computer-based tutorial. J Health Psychol 2003; 8: 149–60. doi:10.1177/1359105303008001450

¹⁹ A virtual coach for learning brief negotiated interview techniques for treating emergency room patients. Proceedings of the spring Symposium on association for the advancement of artificial intelligence (AAAI) 2011.

²⁰ Bickmore TW, Schulman D, Sidner CL. A reusable framework for health counseling dialogue systems based on a behavioral medicine ontology. J Biomed Inform 2011; 44: 183–97. doi:10.1016/j.jbi.2010.12.006

²¹ Richardson CL, Chapman S, White S. Virtual patient educational programme to teach counselling to clinical pharmacists: development and proof of concept. BMJ Stel 2018.

²² Zlotos L, Power A, Hill D, et al. A scenario-based virtual patient program to support substance misuse education. Am J Pharm Educ 2016; 80: 48. doi:10.5688/ajpe80348

²³ Villaume WA, Berger BA, Barker BN. Learning motivational interviewing: Scripting a virtual patient. Am J Pharm Educ 2006; 70: 33. doi:10.5688/aj700233

²⁴ Hubal RC, Kizakevich PN, Guinn CI, et al. The virtual standardized patient. simulated patient-practitioner dialog for patient interview training. Stud Health Technol Inform 2000; 70: 133–8.

²⁵ Sweller J, Ayres P, Kalyuga S. Cognitive load theory. New York: Springer, 2011.

²⁶ Oliver R. Engaging first year students using a Web-supported inquiry-based learning setting. High Educ 2008; 55: 285–301. doi:10.1007/s10734-007-9055-7

²⁷ Jonassen DH, Carr C, Yueh H-P. Computers as mindtools for engaging learners in critical thinking. TechTrends 1998; 43: 24–32. doi:10.1007/BF02818172

²⁸ Yeazel MW, Lindstrom Bremer KM, Center BA. A validated tool for gaining insight into clinicians' preventive medicine behaviors and beliefs: the preventive medicine attitudes and activities questionnaire (PMAAQ). Prev Med 2006; 43: 86–91. doi:10.1016/j.ypmed.2006.03.021

²⁹ Murphy KA, Yeazel M, Center BA. Validity of residents' self-reported cardiovascular disease prevention activities: the preventive medicine attitudes and activities questionnaire. Prev Med 2000; 31: 241–8. doi:10.1006/pmed.2000.0705

³⁰ Amoroso C, Harris MF, Ampt A, et al. The 45 year old health check: feasibility and impact on practices and patient behaviour. Aust Fam Physician 2009; 38: 358–62.

³¹ Chan BC, Jayasinghe UW, Christl B, et al. The impact of a team-based intervention on the lifestyle risk factor management practices of community nurses: outcomes of the community nursing SNAP trial. BMC Health Serv Res 2013; 13: 54. doi:10.1186/1472-6963-13-54

³² Fanaian M, Laws RA, Passey M, et al. Health improvement and prevention study (HIPS) - evaluation of an intervention to prevent vascular disease in general practice. BMC Fam Pract 2010; 11: 57. doi:10.1186/1471-2296-11-57

³³ Waldo J. Motivational interviewing training and provider proficiency. Minnesota State University, 2011.

³⁴ Moyers TB, Manuel JK, Ernst D. Motivational interviewing treatment integrity coding manual 4.1. USA: University of New Mexico, 2015.

³⁵ Moyers TB, Martin T, Manuel JK, et al. Assessing competence in the use of motivational interviewing. J Subst Abuse Treat 2005; 28: 19–26. doi:10.1016/j.jsat.2004.11.001

³⁶ Schoo AM, Lawn S, Rudnik E, et al. Teaching health science students Foundation motivational interviewing skills: use of motivational interviewing treatment integrity and self-reflection to approach transformative learning. BMC Med Educ 2015; 15: 228.

³⁷ Lewis JR. The system usability scale: past, present, and future. Int J Hum Comput Interact 2018; 34: 577–90. doi:10.1080/10447318.2018.1455307

³⁸ Hege I, Dietl A, Kiesewetter J, et al. How to tell a patient’s story? Influence of the case narrative design on the clinical reasoning process in virtual patients. Med Teach 2018; 40: 736–42. doi:10.1080/0142159X.2018.1441985

³⁹ Kistin C, Silverstein M. Pilot studies: a critical but potentially misused component of interventional research. JAMA 2015; 314: 1561–2.

⁴⁰ Bengtsson M. How to plan and perform a qualitative study using content analysis. NursingPlus Open 2016; 2: 8–14. doi:10.1016/j.npls.2016.01.001

⁴¹ Miller WR, Rollnick S. Motivational interviewing: helping people change. 3rd edn. UK: Guildford Press, 2012.

⁴² Arrow P, Raheb J, Miller M. Brief oral health promotion intervention among parents of young children to reduce early childhood dental decay. BMC Public Health 2013; 13: 245. doi:10.1186/1471-2458-13-245

⁴³ Mello MJ, Nirenberg TD, Longabaugh R, et al. Emergency department brief motivational interventions for alcohol with motor vehicle crash patients. Ann Emerg Med 2005; 45: 620–5. doi:10.1016/j.annemergmed.2005.01.026