Introduction
Background
Drug use disorders (DUDs) are prevalent and deadly worldwide [1,2]. Addiction epidemics are worsening—in 2018, 184 people a day died from drug overdose in the US, and the number of fatal-and nonfatal overdoses increased in 2020 [3,4]. However, most people with DUDs do not receive treatment [5,6]. Experts recommend that treatments for DUD should be implemented in primary care to reduce this treatment gap [7-11].
Digital treatments, such as smartphone or tablet app or websites, have been touted as a means to reach more people with effective DUD treatments [12,13]. One way to classify digital treatments is to place them on a spectrum ranging from apps that are used as self-care without the help of a clinician to apps that are fully incorporated into patients’ health care and guided by a clinician [14]. Several clinician-guided treatments for alcohol or drugs are supported by the evidence of efficacy or effectiveness [15-17]. For instance, two platforms—the Therapeutic Educational System and CBT4CBT—deliver cognitive behavioral treatment through web-based modules [15,18]. Both were initially designed by researchers to augment standard care for substance use disorder and have since been marketed to health care systems in the form of apps or websites with patient-facing and clinician-facing features [19,20].
Despite the promise of digital treatments, there is a lack of knowledge about how to optimally integrate them into routine primary care. In trials of digital treatments, including those for DUD and other health conditions, patients often fail to engage with the software, leading to null results [21,22] and failed real-world clinical trials [23]. Without adequate support to enhance motivation to engage with apps, patients prematurely stop using apps or use them rarely, thereby decreasing efficacy [21,22,24-26]. This poses design and implementation problems in primary care. Evidence suggests that patients can benefit from digital treatments if they receive extensive coaching and help to use them. In contrast, the time of patients and clinical teams is limited and filled with competing demands [27-29]. Therefore, the integration of apps into health care will need to balance patients’ desires and needs with these constraints.
Objectives
To incorporate patient voices into the design of approaches for offering apps for DUDs in primary care, this study combined user-centered design methods [30] with qualitative research methods. Drawing from the medical informatics literature, work systems models are often used to guide the design of clinical workflows [31-33]. Central to the concept of work systems’ models are actions, people, and tools that can assist patients and health care teams in embracing health care technologies. We applied these concepts to understand patient preferences on how to introduce DUD treatment apps to patients, assist them with the app setup, and offer appropriate follow-up over time. Consistent with the principles of user-centered design [30], this study served as the first within a series of implementation science studies that will iteratively design and test the effectiveness of approaches for implementing digital treatments for DUDs in real-world health care settings.
Methods
Study Setting
Study participants were recruited from 5 primary care clinics of Kaiser Permanente Washington, a regional integrated health care system, in Seattle area. All clinics employed licensed independent clinical social workers with some training in DUD interventions, and several clinics had primary care providers (PCPs) who prescribed buprenorphine. Consultative addiction psychiatry was available in the health system. Specialist addiction treatment programs were available through an external care network that was contracted by the health system [34].
Eligibility Criteria
The main eligibility criteria, assessed by phone screening, were smartphone use to ensure a basic familiarity with these devices [35] along with the presence of a past-year cannabis, stimulant, or opioid use disorder based on the Mini International Neuropsychiatric Interview for the Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition [36,37]. During our accrual period, we revised the eligibility criteria to also include patients without a past-year opioid use disorder based on Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition if they had been prescribed buprenorphine formulations used to treat opioid use disorder [38].
Participant Identification and Recruitment
Between July 2018 and December 2018, we queried electronic health records to identify patients aged 18 years or more with a recent primary care visit, who had screened positive for cannabis or other drugs [39-42] or had a documented DUD diagnosis. The study staff mailed invitations with a preincentive of US $2 to 101 potentially eligible patients [43,44]. We phoned 77 of these patients for eligibility screening while providing a US $10 incentive for those who completed phone screening, regardless of eligibility. Multimedia Appendix 1 presents a participant recruitment flow diagram noting the reasons for exclusion. We used purposeful sampling [45] to promote sample diversity by race, gender, age, substance type (opioid, cannabis, stimulant), and prior DUD treatment or mutual support program attendance [46].
Sample
The study’s initial recruitment goals were based on pragmatic and empirical criteria. We sought to capture a sufficient diversity of patient perspectives to inform the initial implementation of digital treatments in a primary care practice that could then be iteratively adjusted during implementation within a health care system. We targeted a minimum of 12 participants [47] and stopped recruitment at 14 while monitoring saturation during the analysis [48]. Among the 14 participants, 6 had opioid use disorder or were taking buprenorphine, 9 had cannabis use disorder, and 4 had stimulant use disorder (participant details in Multimedia Appendix 1).
Semistructured Interviews
The interview guide was designed to elicit preferences regarding workflow design among candidate approaches for offering and supporting the delivery of app-based DUD treatments. The interviews utilized user-centered design tools, including personas and storyboards. Personas depicted a visual user profile to show participants an example of a person who might be a typical user of an app [49]. We developed three personas—one each for stimulants, opioids, and cannabis—that depicted a hypothetical patient with several DUD symptoms who was interested in learning about options for help (Multimedia Appendix 1). We presented one persona per participant. We also used storyboards [50], which used illustrated panels to show core features of candidate workflow designs.
The storyboards showed alternative scenarios to support patients’ app use. Scenarios fit into three workflow phases: (1) introducing the app to patients and helping them learn about it (Introduction); (2) setting up patients interested in using the app with the treatment (Setup); and (3) following up with patients who agree to use the app to promote engagement and execute a care plan that includes the app (Follow-up). There were 3 to 4 candidate scenarios as options for each workflow phase (Figure 1; complete storyboards are presented in Multimedia Appendix 1, which includes detailed information about the storyboard design process and rationale behind each scenario). Scenarios varied in whether interactions were done virtually or in person, how much help the patient received, which team members were involved, and other aspects of communication and workflow. Interviewers read aloud all scenarios within a phase, asked participants to rank scenarios according to their preferences, and probed for additional information.
Enlarge this image.Figure 1. Scenarios depicting potential introduction (learning about the app), setup (getting started with the app), and follow-up (getting follow-up while using the app) phases for a drug use disorder treatment app. The boxes denote the overall workflow most preferred by participants. PCP: primary care provider.
Study interview materials were informed by domains of work systems models (eg, workflow actions or tasks, individuals or people, and tools that facilitate work) [33], implementation science frameworks (eg, the extent of support and how to guide patients) [14], and prior digital treatment studies (eg, privacy issues and concerns regarding the use of electronic self-reported data) [22,51-53]. The guide also drew from theory and literature on patient treatment engagement and health services design in primary care [21,22,52-63]. Clinical leaders and clinicians from Kaiser Permanente and a safety net health clinic provided input to the materials. Although not analyzed in this study, the interviews asked about background experiences (eg, app use; Multimedia Appendix 1 contains an interview guide summary). Interviews were conducted in person and digitally recorded and professionally transcribed. Interviews lasted for an average of 87 minutes, including informed consent.
The Kaiser Permanente Washington Institutional Review Board granted ethical approvals. Written informed consent was obtained from all the participants enrolled in this study. A waiver of written informed consent was obtained to conduct telephone screenings. A waiver of informed consent was obtained for the identification of potential participants using electronic health records.
Data Analysis
For qualitative analysis, we imported transcripts into Dedoose v7.0.23 (SocioCultural Research Consultants) [64]. We applied template analyses to qualitatively code excerpts of text [65], which used a combination of inductive and deductive approaches. A priori codes were based on the interview guide topics. Codebook definitions were informed posthoc by the Workflow Elements Model [31,32] constructs to help in classifying participant preferences regarding the physical or virtual actions that need to be performed, the people (eg, clinicians) who need to perform the actions, and the physical and virtual tools used to promote app delivery and use. For the first 5 transcripts, 2 study team members coded them, a third study team member reconciled them, and the team discussed achieving consensus in the form of a codebook. To further increase rigor, we kept memos and an audit trail of code revisions [66,67]. The remaining transcripts were split among team members for coding, and the principal investigator (PI; JEG) reconciled them. The lead analyst (TEM) examined the fully coded data for patterns within the coding hierarchy and preliminary themes that occurred across codes, regularly meeting with the PI to review results. In several meetings, the PI and lead analyst used affinity mapping, a user-centered design activity that helps in clustering similar ideas or concepts together [68] to further refine the themes. The themes were discussed with other team members before they were finalized.
Availability of Data and Materials
The interview materials are provided in Multimedia Appendix 1. Additional interview materials are available from the first author upon reasonable request.
Results
Preferences Regarding Workflow Actions
This section presents patient preferences regarding the actions performed while facilitating a DUD treatment app, by each workflow stage.
Introduction Stage: Learning About the Treatment App
Nearly all participants preferred to learn about app-based treatment options during an in-person primary care visit where drug use was already being discussed. Participants generally did not want to learn about an app during an unexpected cold call with a clinician they had not met. Cold calls in relation to drug use would feel intrusive, potentially awkward, and embarrassing. Participants who were open to receiving a phone call to learn about an app said they would at least need to know about the phone call in advance. Only 2 participants were interested in a self-guided treatment where they would learn about an app via a pamphlet without any interaction with a health care professional, and only 1 participant said that they would respond to a waiting room advertisement. A few noted that the convenience of accessing the app without help from a clinician was appealing, but others said they would not be motivated to use an app without support. As one participant explained:
I just don’t see that I would pick [the flyer] up ... It would be more effective if, after meeting with my primary care person about my addiction issues, they got me in direct contact with [a mental health clinician]. [P3, who was receiving medication treatment for opioid use disorder]
Setup Stage: Getting Started With the App
Overall, participants thought that it was critical to avoid getting “stuck” while setting up and starting to use an app. Having a seamless experience would be critical for maintaining motivation. One participant said the following:
Sit next to me and walk me through it. Hold my hand. [P13, participant with cannabis use disorder]
Most participants preferred in-person assistance to help them get started with the app once they decided to use it. However, several others wanted to receive this help over the phone to reduce the amount of time spent at their visit or to avoid returning for a visit. For these participants, the phone setup was “the best of both worlds.” One participant said as follows:
number one is like, we agree that [persona name] will schedule an in-person appointment. In my case I’m very reluctant to do stuff like that, so okay, extra work for me–awesome, thanks. This one [solo setup scenario] is like all on my own, no support. So this one [phone setup scenario] I feel like is the best of both worlds. [P3, participant with cannabis use disorder]
Still, some thought that scheduling phone appointments would be a chore. Few preferred to set up the app without help; these participants noted that setting up the app on their own would save time.
Follow-up Stage: Getting Support Over Time for Engagement and Care With an App
Half of the participants preferred follow-up to occur over the phone. This would provide personalized communication and a relationship that would help them stay accountable without the inconvenience and monetary costs of an in-person visit. Even those who favored an initial visit in person generally preferred a phone follow-up. Few preferred a follow-up strategy that only involved secure messages, but they liked the idea of receiving messages in addition to phone calls. One participant noted:
Secure messages and phone for me ... Because that way you have somebody on the phone to tell you if you’ve interpreted the message right ... I’ve had that happen many times. A doctor will say something and I don’t necessarily interpret it correctly or the way that it’s meant. [P14, participant with opioid and stimulant use disorder]
Most participants thought that reading and responding to a message at a later time would be convenient, and some were more receptive to more frequent contacts if done over messages. However, others noted that they do not check messages or thought they would be “too easy to ignore.” One participant added the following:
If you’re having that reluctance, that’s so much easier to just be like well, no, I’m not going to do that. [P10, participant with cannabis use disorder]
Preferences Regarding the People Who Could Deliver Digital Treatments
Preferences regarding the people who could facilitate a DUD treatment app were often applicable across multiple stages of the workflow (Multimedia Appendix 1). Thus, the preferences are presented here for the overall workflow.
Most participants preferred a clinician with mental health expertise to introduce and guide them in app use over time. Several participants said that mental health clinicians might have more knowledge about treatment options than a PCP, more experience treating drug use, or more time to describe features of an app and walk them through the setup process. However, some preferred to only talk to PCPs and said that mental health clinicians would make it feel like their “issue was serious.” Other benefits of working with a PCP ranged from wanting to work with someone with whom they had an established rapport, wanting fewer clinicians involved in their care, and wanting to work with someone who can offer medical advice. Many acknowledged logistical constraints of working with a PCP (eg, limited time and challenge of scheduling appointments). One participant said the following:
I don’t think the primary care [provider] needs to be concerned with that. They need to have some knowledge of it, but I think the major, the main focus of it would be with the social worker, plus the social worker would be able to follow up with the patient as far as once they get started using the app. It would be easier follow up for the social worker than the primary care [provider]. [P5, participant with stimulant use disorder]
A few participants said it would be helpful to hear from someone with lived experience who had “been through it before” to help them decide whether to use the app over other treatment options. They did not feel that it was important to have such an individual remain involved after they had started using the app.
Finally, participants described the value of having access to technical support for help with downloading, setup, and use of the app. Participants pointed out that they would not want technical issues to consume valuable visit time. Participants clearly differentiated technical or setup assistance from treatment-related assistance and drew boundaries around the type of help they would want from each person.
Preferences Regarding the Tools That Could Help Facilitate the Use of the Digital Treatment
Participants described a range of ideas about virtual or physical tools that could facilitate the delivery of an app (Textbox 1). For instance, pamphlets, user ratings and reviews, and trial versions of the app were suggested as tools for the introduction stage. Written instructions and video tutorials could help with the setup stage. Several tools were suggested for the follow-up stage to facilitate communication (eg, a “Get Support” button to help contact the care team).
Having a way to check in with your doctor about it would be useful ... In [the phone setup scenario] if there could be a way to demonstrate it like in [the in-person scenario], with a screen sharing thing – I think that would be super useful, or maybe like a video, like an instructional video showing how to navigate it, that would be useful, since you wouldn’t be able to do it in person. [P12, participant with cannabis and stimulant use disorder]
Tools that could help facilitate the use of a drug use disorder treatment app in primary care, as suggested by patients. Tools are presented by the workflow stage.
Introduction Stage: Learning About the Treatment App
Setup Stage: Getting Started With the App
Follow-up Stage: Getting Support Over Time for Engagement and to Execute a Care Plan While Using the App
Cross-Cutting Themes
Four cross-cutting themes emerged across codes in the codebook. Themes are described below; Textbox 2 provides representative quotes for each theme (Multimedia Appendix 1).
Themes derived from analyses across the codebook: cross-cutting recommendations for designing a patient-centered approach for offering treatment apps for drug use disorders in primary care
Established Relationships and Trust Would Facilitate a Better Patient Experience
Patients Were Open to Team-Based Approach When Getting Support From Their Primary Care Team in the Use of Apps
Patients Felt a Tension Between Effectiveness and Convenience in Aspects of Workflow Design
The Workflow Needs to Meet Patients Where They Are At
Established Relationships and Trust
Participants told us that they placed value in having a connection with the health care professional who worked with them on using a digital treatment. It would be ideal if this would be someone who had already established a relationship with them, regardless of their clinical role. Even if there was no pre-existing relationship, it was critical for this person to be compassionate and caring because conversations about substance use can be stigmatizing or embarrassing.
Openness to a Team-Based Approach
Although participants had a desire to work with someone with whom they had an existing relationship, the trust between a PCP and a patient appeared to extend to the broader primary care team. That is, a PCP would not necessarily be the one holding [their] hand throughout using an app. Some noted that PCPs are busy and often delegate care to other team members, including DUD care. However, there were some important bounds around this division of responsibility. For instance, having a “third party” from outside the health system support patients was seen as a bad idea.
An important exception to this theme is that a participant firmly wanted to work with one clinician. Needing to talk to an additional person about an app could be an “extra step” and could “add like the risk that they won’t do it at all.”
Tension Between Effectiveness and Convenience
Participants described the pros and cons of the intensity of hypothetical workflow interactions. Regarding the general workflow design, they noted that convenience and effectiveness were in opposition. In-person visits were more “personable” and would hold them more “accountable” to using an app; however, an in-person visit would require more time, travel, and potentially copays. Secure messages, or receiving reminders through the app, were considered convenient because participants could read and respond whenever they wanted; however, they acknowledged that these messages might not be as powerful as a phone call or face-to-face visit in keeping them engaged.
The Workflow Needs to Meet the Patient Where They Are At
Participants suggested that workflows should be tailored to individuals depending on their level of motivation, how often they use substances, and how long or how successfully they have been managing their substance use. For instance, this desire for tailoring reflected that someone with less motivation may need more hands-on help, someone who is using substances daily might want to be contacted every day, and someone who is just beginning to address their use and/or who is actively struggling may need more frequent follow-ups.
Synthesis
Upon synthesis of the qualitative data, and considering the number of participants who preferred each storyboard scenario (Multimedia Appendix 1), we suggest a general approach for facilitating the use of DUD treatment apps, which is described using the story of a fictitious patient (Table 1). Briefly, once the patient expresses interest in using an app for DUD, the PCP confirms their willingness to talk with a clinician with mental health expertise that will help them get started and keep them accountable while initiating treatment. The clinician would teach the patient about the app, help them plan for using the app, agree on a structure for follow-up, and then provide regular follow-up by telephone and messaging (bold boxes, Figure 1). The clinician would further build rapport and ask the patient their preferred cadence, method of follow-up, and modality of contact (eg, in person vs phone). Importantly, the specific approach would remain flexible and tailored to the patient.
Table 1. A general approach for supporting patients in using a drug use disorder treatment app in primary care, based on participant preferences.
| Workflow stage | Hypothetical experience for a fictitious patient Cory | Why participants liked this experience | Other experiences preferred by participants |
| Introduction stage: learning about the treatment app | Cory completes an annual health screen that asks about alcohol and drug use. Cory’s PCPa expresses concern that her regular substance use could affect her health. Cory is interested in learning about options that could help her change. She agrees to talk to a mental health clinician on the primary care team. Privately, Cory and the mental health clinician discuss Cory’s goals for change, and review a few different options, including a treatment app for drug use. | Participants wanted to discuss substance use with a provider they already knew—but they also recognized that their PCP might not have the time or right expertise. Being seamlessly connected to a mental health clinician would feel “more personal” and provide support beyond what their PCP could offer. | Participants who wanted to talk to only one person—usually their PCP—said they would try to set up the treatment app on their own after their PCP ordered it. |
| Setup stage: getting started with the app | A mental health clinician describes features and content of the DUDb treatment app that might be helpful to Cory. They give Cory instructions for how to get started, and they agree to check in after a couple weeks. | Most participants wanted to learn from someone on their care team how an app would benefit them and how to use it. Chances of using the app would be higher if these were discussed when motivation was high. | Several participants felt comfortable getting started with a treatment app on their own. Some said that technical support would be necessary. |
| Follow-up stage: getting support over time for engagement and executing a care plan while using the app | Cory gives the app a try. She eventually stops using the app after a couple weeks. However, she re-engages with the app after exchanging secure messages with her mental health clinician that covers a status update, tailored recommendations for using the app, and plans for a follow-up phone call. | Participants said that phone follow-up offered more support than secure messages and placed fewer demands on their time or finances than an in-person visit. Many wanted follow-up spaced out over time to help hold them accountable to using the app. | Benefits of follow-up via secure message include the choice in when and how often to responded to messages (unlike visits or phone calls). In-person appointments would be reserved for additional support and accountability. |
aPCP: primary care provider.
bDUD: drug use disorder.
Discussion
Principal Findings
This study conveyed patient perspectives on the use of apps as part of treatment for DUD in primary care. Overall, participants desired to receive support from their health care teams in using apps and voiced little interest in using them without clinician guidance. There was a consensus among participants that they preferred to work with a trusted, competent clinician who could guide them in using the app over time. They stressed the importance of follow-up and felt that in most cases, this could generally be done through telephone with the addition of asynchronous secure messaging whenever needed.
These findings contribute novel information about patient preferences, laying the groundwork for research on the implementation of apps for DUDs in health care. Patients desire low-barrier, nonstigmatizing interventions for DUD in primary care [69,70]. Apps could potentially help address this gap in care. Although prior clinical trials have relied on research staff to “train” patients in using apps or facilitate their ongoing use [15,21,71-74], future studies can use these findings to inform the involvement of health care teams, instead of researchers, in various aspects of app delivery and implementation.
Overall, participants expressed that the most ideal way to offer apps for DUDs was during routine clinical interactions regarding drug use. This approach enables several aspects of care that participants desired, including the ability to build trust with a clinician, obtain medical and mental health advice, and go home with an appropriate app and clear expectations about treatment. The literature has described an alternative approach, where apps are offered as a self-guided option in the absence of clinician guidance [14,75], potentially reducing health care costs and burden. Such an approach was generally not preferred by participants in this study; some wanted access to a self-care option, but clinician-guided care was viewed as more effective and engaging. The findings are consistent with the literature that describes the need for supportive accountability while delivering app-based treatments, where patients establish a relationship with a helper that, among other things, sets expectations for app use and provides assistance over time [76]. We note that this study excluded participants whose drug use did not lead to a DUD; perhaps, self-care options should be further studied among patients with lower drug use severity.
Participants felt that treatment with an app should be seamless, free of technical glitches, and other barriers that could decrease their motivation or lead to “getting stuck.” Apps and associated clinical workflows need to impose few barriers. Indeed, many people with DUD report logistical barriers to treatment [77-82]. Some participants were also concerned about copays, which contributed to their support preferences. Future studies should design app implementation approaches that address socioeconomic and other barriers that could lead to the inequitable provision of these treatments [83,84].
The literature has also highlighted potential logistical challenges from a health system perspective. For instance, PCPs are heavily burdened and busy [27-29]. Indeed, a prior implementation study of a DUD app had to modify its initial plans by not involving PCPs [51]. Fortunately, participants in this study acknowledged these difficulties and were open to working with team members other than their PCP. Some studies in primary care failed to adequately engage patients in app use over time [21,22], and/or have been halted because workflows were not adequately developed to reach and communicate with patients [23]. This study advances the literature by adding patient voices that can inform future research on app delivery and implementation in routine care.
Limitations
This study has several limitations. The sample size was small, which limits generalizability; however, we followed the standard of saturation in qualitative research. Furthermore, our use of purposeful sampling promoted a rich accounting of a diversity of patient perspectives and preferences across important demographic and substance use characteristics to inform the implementation of apps. Nonetheless, the small sample size limited our ability to identify the ideas and preferences of all possible patients. Although there was saturation in the analysis of workflow design principles and cross-cutting themes (eg, new data analyzed were repetitious of prior data and there was no indication of new emerging themes), the ideas regarding tools remained diverse. In this case, we opted to present a table that captured all participants’ ideas. Future study iterations are driven by additional data collection. Participants were from 5 clinics of a single integrated health care system in the United States from a single geographic region. Participants may have been more accustomed to working with multiple care providers than patients who received care in health systems with little care coordination. Thus, the results might not translate to patients from other groups. Although the goal of this study was to understand patient perspectives, clinicians’ voices were not assessed, which is also a limitation. This study’s focus on patients was driven by knowledge that variations in patient engagement impacted the results of trials of digital treatments [21,22,24-26,73]. Future research should elicit clinician perspectives on the mechanics of delivering apps for DUDs in primary care.
Conclusions
The perspectives of primary care patients with cannabis, opioid, and stimulant use disorders suggest that a user-centered implementation of DUD treatment apps in primary care will require health systems to guide and support patients. Research is needed to evaluate clinician perspectives on workflows for delivering apps and to test the feasibility of the design considerations suggested by participants in this study. One such study is our ongoing implementation research trial funded by the National Institute on Drug Abuse (award number R01DA047954), which uses a randomized factorial design to evaluate different approaches for implementing a digital treatment for substance use disorders, while concurrently collecting qualitative data on implementation and workflow design. As more emphasis is placed on the use of digital treatments for DUDs for primary care patients, convenient methods for engaging patients and supporting them before, during, and after treatment will be paramount.
Acknowledgments
The authors are grateful to the patients of Kaiser Permanente Washington for volunteering for this study. The authors would also like to thank Dr Manu Thakral at the University of Massachusetts Boston for providing input on workflow appropriateness and feasibility and Kristen Delaney and Kay Theis at Kaiser Permanente Washington Health Research Institute for their expert database and electronic medical records programming that made this study possible. Finally, the authors thank the faculty and fellows of the National Institutes of Health Implementation Research Institute and the Mixed Methods Research Training Programs for their feedback on this work. The primary funding for this study was obtained through a pilot award to JEG from the Kaiser Permanente Washington Research Institute Development Fund. JEG and KAB were awarded funding by the National Institute on Alcohol Abuse and Alcoholism under K01AA023859 and K24AA022128. This was also supported by the National Institute on Drug Abuse under award R01DA047954. GMC is supported by the Translational Research Institute UL1TR003107 through the National Center for Advancing Translational Sciences of the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Authors' Contributions
Funding acquisition and the first manuscript draft was done by JEG. Study conceptualization and design was conducted by JEG, TEM, CL, ALH, RP, RMC, AGM, GMC, and KAB. Data acquisition was carried out by JEG, TEM, CL, KK, AKL, TB, RP, and RMC. Data analysis was done by TEM, JEG, CL, AKL, and GMC. All authors were involved in editing and manuscript approval.
Conflicts of Interest
Several authors (JEG, RP, RMC, KAB) are investigators on a trial funded by the National Institute on Drug Abuse (R01DA047954; PI: Glass), which evaluates the implementation of reSET and reSET-O that are digital therapeutics for substance use disorder marketed by Pear Therapeutics, Inc. Pear Therapeutics Inc provided digital therapeutic prescriptions at no cost to Kaiser Permanente Washington during a quality improvement pilot study that precedes the implementation trial. JEG is a coinvestigator on a Small Business Innovation Award funded awarded to Pear Therapeutics, Inc by the National Institute on Drug Abuse, which evaluates potential improvements to reSET-O (R44DA042652). This study is informing our research on the implementation of digital therapeutics, but Pear Therapeutics, Inc was not involved in this study and does not provide funding to the authors.
Multimedia Appendix 1
Interview materials and additional results for the study. Approaches for implementing app-based digital treatments for drug use disorders into primary care: A qualitative, user-centered design study.
Dwyer-Lindgren L, Bertozzi-Villa A, Stubbs RW, Morozoff C, Shirude S, Unützer J, et al. Trends and patterns of geographic variation in mortality from substance use disorders and intentional injuries among US counties, 1980-2014. J Am Med Assoc 2018 Mar 13;319(10):1013-1023
US Burden of Disease Collaborators, Mokdad AH, Ballestros K, Echko M, Glenn S, Olsen HE, et al. The state of US Health, 1990-2016: burden of diseases, injuries, and risk factors among US states. J Am Med Assoc 2018 Dec 10;319(14):1444-1472.
Hedegaard H, Miniño AM, Warner M. Drug overdose deaths in the United States, 1999–2018. Center for Disease Control and Prevention. Atlanta, GA: Center for Disease Control and Prevention; 2020. URL: https://www.cdc.gov/nchs/products/databriefs/db356.htm [accessed 2020-09-01]
The consequences of COVID-19 on the overdose epidemic: overdoses are increasing. Overdose Detection Mapping Application Program (ODMAP). 2020. URL: http://odmap.org/Content/docs/news/2020/ODMAP-Report-May-2020.pdf [accessed 2021-05-24]
Grant BF, Goldstein RB, Saha TD, Chou SP, Jung J, Zhang H, et al. Epidemiology of DSM-5 alcohol use disorder: results from the National Epidemiologic Survey on alcohol and related conditions III. JAMA Psychiatry 2015 Aug;72(8):757-766
Grant BF, Saha TD, Ruan WJ, Goldstein RB, Chou SP, Jung J, et al. Epidemiology of DSM-5 drug use disorder: results from the National Epidemiologic Survey on alcohol and related conditions-III. JAMA Psychiatry 2016 Jan;73(1):39-47
Ghitza UE, Tai B. Challenges and opportunities for integrating preventive substance-use-care services in primary care through the Affordable Care Act. J Health Care Poor Underserved 2014 Feb;25(1 Suppl):36-45
Saitz R, Larson MJ, Labelle C, Richardson J, Samet JH. The case for chronic disease management for addiction. J Addict Med 2008 Jun;2(2):55-65
Crowley RA, Kirschner N, Health and Public Policy Committee of the American College of Physicians. The integration of care for mental health, substance abuse, and other behavioral health conditions into primary care: executive summary of an American College of Physicians position paper. Ann Intern Med 2015 Aug 18;163(4):298-299.
Institute of Medicine. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, D.C., United States: National Academy Press; 2006:1-204.
National Council for Behavioral Health. URL: https://healthinsight.org/tools-and-resources/download/123-educational-resources/1535-implementing-care-for-alcohol-other-drug-use-in-medical-settings-an-extension-of-sbirt [accessed 2021-05-24]
Quanbeck AR, Gustafson DH, Marsch LA, McTavish F, Brown RT, Mares M, et al. Integrating addiction treatment into primary care using mobile health technology: protocol for an implementation research study. Implement Sci 2014 May 29;9(1):65
Marsch LA, Campbell A, Campbell C, Chen C, Ertin E, Ghitza U, et al. The application of digital health to the assessment and treatment of substance use disorders: the past, current, and future role of the National Drug Abuse Treatment Clinical Trials Network. J Subst Abuse Treat 2020 Mar;112S:4-11
Hermes ED, Lyon AR, Schueller SM, Glass JE. Measuring the implementation of behavioral intervention technologies: recharacterization of established outcomes. J Med Internet Res 2019 Jan 25;21(1):e11752
Campbell AN, Nunes EV, Matthews AG, Stitzer M, Miele GM, Polsky D, et al. Internet-delivered treatment for substance abuse: a multisite randomized controlled trial. Am J Psychiatry 2014 Jun;171(6):683-690
Gustafson DH, McTavish FM, Chih M, Atwood AK, Johnson RA, Boyle MG, et al. A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry 2014 May;71(5):566-572
Kaner EF, Beyer FR, Garnett C, Crane D, Brown J, Muirhead C, et al. Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. Cochrane Database Syst Rev 2017 Dec 25;9:CD011479.
Kiluk BD, Nich C, Buck MB, Devore KA, Frankforter TL, LaPaglia DM, et al. Randomized clinical trial of computerized and clinician-delivered CBT in comparison with standard outpatient treatment for substance use disorders: primary within-treatment and follow-up outcomes. Am J Psychiatry 2018 Sep 01;175(9):853-863
Pear Therapeutics, Inc., Boston. 2019 Mar 2. URL: https://peartherapeutics.com/products/reset-reset-o/ [accessed 2020-03-02]
Scientific evidence is at the heart of chess. Chess Health. 2021. URL: https://www.chess.health/evidence/ [accessed 2021-02-17]
Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. Br Med J 2015;351:h5627
Knowles SE, Lovell K, Bower P, Gilbody S, Littlewood E, Lester H. Patient experience of computerised therapy for depression in primary care. BMJ Open 2015 Nov 30;5(11):e008581.
Thies K, Anderson D, Cramer B. Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: interview analysis of staff and patients in a failed randomized trial. JMIR Hum Factors 2017 Oct 03;4(4):e24
Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother 2012 Jul;12(7):861-870.
Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205.
Dedert EA, McDuffie JR, Stein R, McNiel JM, Kosinski AS, Freiermuth CE, et al. Electronic interventions for alcohol misuse and alcohol use disorders: a systematic review. Ann Intern Med 2015 Aug 4;163(3):205-214.
Chernof BA, Sherman SE, Lanto AB, Lee ML, Yano EM, Rubenstein LV. Health habit counseling amidst competing demands: effects of patient health habits and visit characteristics. Med Care 1999 Aug;37(8):738-747.
Jaén CR, Stange K, Nutting P. Competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract 1994 Feb;38(2):166-171.
Rost K, Nutting P, Smith J, Coyne J, Cooper-Patrick L, Rubenstein L. The role of competing demands in the treatment provided primary care patients with major depression. Arch Fam Med 2000 Feb;9(2):150-154.
Ergonomics of Human-System Interaction - Part 210: Human-centred design for interactive systems. Industry Standards Organization. 2019. URL: https://landingpage.bsigroup.com/LandingPage/Undated?UPI=000000000030388991 [accessed 2021-05-24]
Mickelson RS, Unertl KM, Holden RJ. Medication management: the macrocognitive workflow of older adults with heart failure. JMIR Hum Factors 2016 Oct 12;3(2):e27
Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc 2010;17(3):265-273
Carayon P. The balance theory and the work system model… twenty years later. Int J Hum-Comput Int 2009 Jun 08;25(5):313-327.
Glass JE, Bobb JF, Lee AK, Richards JE, Lapham GT, Ludman E, et al. Study protocol: a cluster-randomized trial implementing Sustained Patient-centered Alcohol-related Care (SPARC trial). Implement Sci 2018 Aug 06;13(1):108.
McClure JB, Hartzler AL, Catz SL. Design considerations for smoking cessation apps: feedback from nicotine dependence treatment providers and smokers. JMIR Mhealth Uhealth 2016;4(1):e17
Sheehan D, Lecrubier Y, Sheehan KH, Janavs J, Weiller E, Keskiner A, et al. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry 1997;12(5):232-241.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59 Suppl 20:22-34.
Boudreau DM, Lapham G, Johnson EA, Bobb JF, Matthews AG, McCormack J, et al. Documented opioid use disorder and its treatment in primary care patients across six U.S. health systems. J Subst Abuse Treat 2020 Mar;112S:41-48
Lapham GT, Lee AK, Caldeiro RM, Glass JE, Carrell DS, Richards JE, et al. Prevalence of behavioral health conditions across frequency of cannabis use among adult primary care patients in Washington State. J Gen Intern Med 2018 Nov;33(11):1833-1835
Lapham GT, Lee AK, Caldeiro RM, McCarty D, Browne KC, Walker DD, et al. Frequency of cannabis use among primary care patients in Washington State. J Am Board Fam Med 2017;30(6):795-805
Richards JE, Bobb JF, Lee AK, Lapham GT, Williams EC, Glass JE, et al. Integration of screening, assessment, and treatment for cannabis and other drug use disorders in primary care: an evaluation in three pilot sites. Drug Alcohol Depend 2019 Aug 01;201:134-141
Sayre M, Lapham GT, Lee AK, Oliver M, Bobb JF, Caldeiro RM, et al. Routine assessment of symptoms of substance use disorders in primary care: prevalence and severity of reported symptoms. J Gen Intern Med 2020 Apr;35(4):1111-1119.
Edwards P, Cooper R, Roberts I, Frost C. Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. J Epidemiol Community Health 2005 Nov;59(11):987-999
Doody MM, Sigurdson AS, Kampa D, Chimes K, Alexander BH, Ron E, et al. Randomized trial of financial incentives and delivery methods for improving response to a mailed questionnaire. Am J Epidemiol 2003 Apr 01;157(7):643-651.
Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health 2015 Sep;42(5):533-544
Hasin DS, Greenstein E, Aivadyan C, Stohl M, Aharonovich E, Saha T, et al. The Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5): procedural validity of substance use disorders modules through clinical re-appraisal in a general population sample. Drug Alcohol Depend 2015 Mar 01;148:40-46
Virzi RA. Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 2016 Nov 23;34(4):457-468.
Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant 2018;52(4):1893-1907
Adlin T, Pruitt J. The Essential Persona Lifecycle: Your Guide to Building and Using Personas. Burlington, Massachusetts, United States: Morgan Kaufmann; 2010:1-10.
Truong K, Hayes G, Abowd G. Storyboarding: an empirical determination of best practices and effective guidelines. In: Proceedings of the 6th conference on Designing Interactive systems. 2006 Presented at: DIS06: Designing Interactive Systems 2006; June 26-28, 2006; University Park PA USA p. 12-21.
Mares M, Gustafson DH, Glass JE, Quanbeck A, McDowell H, McTavish F, et al. Implementing an mHealth system for substance use disorders in primary care: a mixed methods study of clinicians' initial expectations and first year experiences. BMC Med Inform Decis Mak 2016 Sep 29;16(1):126
Lapham GT, Hawkins EJ, Chavez LJ, Achtmeyer CE, Williams EC, Thomas RM, et al. Feedback from recently returned veterans on an anonymous web-based brief alcohol intervention. Addict Sci Clin Pract 2012 Aug 28;7:17
O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak 2016 Sep 15;16(1):120
Jochems EC, Mulder CL, Duivenvoorden HJ, van der Feltz-Cornelis CM, van Dam A. Measures of motivation for psychiatric treatment based on self-determination theory: psychometric properties in Dutch psychiatric outpatients. Assessment 2014 Aug 02;21(4):494-510.
Ryan RM, Plant RW, O'Malley S. Initial motivations for alcohol treatment: relations with patient characteristics, treatment involvement, and dropout. Addict Behav 1995 May;20(3):279-297.
Urbanoski KA, Wild TC. Assessing self-determined motivation for addiction treatment: validity of the Treatment Entry Questionnaire. J Subst Abuse Treat 2012 Jul;43(1):70-79.
Cloud R, Kingree J. Concerns about dose and underutilization of twelve-step programs: models, scales, and theory that inform treatment planning. Recent Dev Alcohol 2008;18:283-301.
Wild TC, Cunningham JA, Ryan RM. Social pressure, coercion, and client engagement at treatment entry: a self-determination theory perspective. Addict Behav 2006 Oct;31(10):1858-1872.
Wild T, Newton-Taylor B, Alletto R. Perceived coercion among clients entering substance abuse treatment: structural and psychological determinants. Addict Behav 1998 Jan;23(1):81-95.
Bauer AM, Rue T, Keppel GA, Cole AM, Baldwin L, Katon W. Use of mobile health (mHealth) tools by primary care patients in the WWAMI region Practice and Research Network (WPRN). J Am Board Fam Med 2014;27(6):780-788
Hartzler AL, BlueSpruce J, Catz SL, McClure JB. Prioritizing the mHealth design space: a mixed-methods analysis of smokers' perspectives. JMIR Mhealth Uhealth 2016 Aug 05;4(3):e95
McClure JB, Anderson ML, Bradley K, An LC, Catz SL. Evaluating an adaptive and interactive mhealth smoking cessation and medication adherence program: a randomized pilot feasibility study. JMIR Mhealth Uhealth 2016 Aug 03;4(3):e94
Andréasson S, Danielsson A, Wallhed-Finn S. Preferences regarding treatment for alcohol problems. Alcohol Alcohol 2013;48(6):694-699.
Dedoose Version 8.0.35, web application for managing, analyzing, and presenting qualitative and mixed method research data. SocioCultural Research Consultants, Los Angeles, CA2018. URL: https://www.dedoose.com/ [accessed 2021-05-24]
King N. Using templates in the thematic analysis of text. In: Cassell C, Symon G, editors. Essential Guide to Qualitative Methods in Organizational Research. Thousand Oaks, California, United States: SAGE Publications Ltd; 2004:256-270.
Creswell J. Qualitative Inquiry and Research Design: Choosing Among Five Approaches, 3rd Ed. Thousand Oaks, CA: SAGE Publications; 2013:1-448.
Iversen RR. Book Review: Padgett, D. K. (2008). Qualitative Methods in Social Work Research (2nd ed.). Thousand Oaks, CA: Sage. 304 pp., $49.95 (paperback), ISBN 978-1-4129-5193-7; $84.95 (hardbound), ISBN 978-1-4129-5192-0. Res Soc Work Pract 2009 Dec 03;20(1):125-126.
Beyer H, Holtzblatt K. Contextual Design. San Francisco, CA: Morgan Kaufman; 1998:1-496.
Wallhed FS, Bakshi A, Andréasson S. Alcohol consumption, dependence, and treatment barriers: perceptions among nontreatment seekers with alcohol dependence. Subst Use Misuse 2014 May;49(6):762-769.
Barry CL, Epstein AJ, Fiellin DA, Fraenkel L, Busch SH. Estimating demand for primary care-based treatment for substance and alcohol use disorders. Addiction 2016 Aug;111(8):1376-1384
Christensen DR, Landes RD, Jackson L, Marsch LA, Mancino MJ, Chopra MP, et al. Adding an internet-delivered treatment to an efficacious treatment package for opioid dependence. J Consult Clin Psychol 2014 Dec;82(6):964-972
Bennett GG, Steinberg D, Askew S, Levine E, Foley P, Batch BC, et al. Effectiveness of an app and provider counseling for obesity treatment in primary care. Am J Prev Med 2018 Dec;55(6):777-786
Gilbody S, Brabyn S, Lovell K, Kessler D, Devlin T, Smith L, REEACT collaborative. Telephone-supported computerised cognitive-behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial. Br J Psychiatry 2017 May;210(5):362-367.
Graham AK, Greene CJ, Kwasny MJ, Kaiser SM, Lieponis P, Powell T, et al. Coached mobile app platform for the treatment of depression and anxiety among primary care patients: a randomized clinical trial. JAMA Psychiatry 2020 Sep 01;77(9):906-914.
Muñoz RF. The efficiency model of support and the creation of digital apothecaries. Clin Psychol: Sci Pract 2017 Mar;24(1):46-49.
Mohr DC, Duffecy J, Jin L, Ludman EJ, Lewis A, Begale M, et al. Multimodal e-mental health treatment for depression: a feasibility trial. J Med Internet Res 2010 Dec 19;12(5):e48
Perron BE, Mowbray OP, Glass JE, Delva J, Vaughn MG, Howard MO. Differences in service utilization and barriers among Blacks, Hispanics, and Whites with drug use disorders. Subst Abuse Treat Prev Policy 2009 Mar 13;4(1):-.
Copeland J. A qualitative study of barriers to formal treatment among women who self-managed change in addictive behaviours. J Subst Abuse Treat 1997;14(2):183-190.
Cunningham JA, Sobell LC, Sobell MB, Agrawal S, Toneatto T. Barriers to treatment: why alcohol and drug abusers delay or never seek treatment. Addict Behav 1993 May;18(3):347-353.
Rapp RC, Xu J, Carr CA, Lane DT, Wang J, Carlson R. Treatment barriers identified by substance abusers assessed at a centralized intake unit. J Subst Abuse Treat 2006 Apr;30(3):227-235
Sexton RL, Carlson RG, Leukefeld CG, Booth BM. Barriers to formal drug abuse treatment in the rural south: a preliminary ethnographic assessment. J Psychoactive Drugs 2008 Jun;40(2):121-129.
van der Pol P, Liebregts N, de Graaf R, Korf DJ, van den Brink W, van Laar M. Facilitators and barriers in treatment seeking for cannabis dependence. Drug Alcohol Depend 2013 Dec 01;133(2):776-780.
Cheng C, Beauchamp A, Elsworth GR, Osborne RH. Applying the electronic health literacy lens: systematic review of electronic health interventions targeted at socially disadvantaged groups. J Med Internet Res 2020 Aug 13;22(8):e18476
Fang ML, Canham SL, Battersby L, Sixsmith J, Wada M, Sixsmith A. Exploring privilege in the digital divide: implications for theory, policy, and practice. Gerontologist 2019 Jan 09;59(1):1-15.
© 2021. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Matson, Theresa E
; Lim, Catherine
; Hartzler, Andrea L
; Kimbel, Kilian
; Lee, Amy K
; Beatty, Tara
; Parrish, Rebecca
; Caldeiro, Ryan M
; Angela Garza McWethy
; Curran, Geoffrey M
; Bradley, Katharine A