“It reminds me and motivates me”: Human-centered design and implementation of an interactive, SMS-based digital intervention to improve early retention on antiretroviral therapy: Usability and acceptability among new initiates in a high-volume, public clinic in Malawi

Abstract

Background

Early retention of people living with HIV (PLHIV) in antiretroviral therapy (ART) programs is critical to improve individual clinical outcomes and viral load suppression. Although many mobile health (mHealth) interventions aim to improve retention in care, there is still lack of evidence on mHealth success or failure, including from patient’s perspectives. We describe the human-centered design (HCD) process and assess patient usability and acceptability of a two-way texting (2wT) intervention to improve early retention among new ART initiates at Lighthouse Trust clinic in Lilongwe, Malawi.

Methods

An iterative HCD approach focused on patient and provider users’ needs, incorporating feedback from multidisciplinary teams to adapt 2wT for the local, public clinic context. We present mixed-methods usability and acceptability results from 100 participants, 50 at 3-months and 50 at 6-months, post 2wT enrollment, and observations of these same patients completing core tasks of the 2wT system.

Results

Among the 100 usability respondents, 95% were satisfied with visit reminders, and 88% would recommend reminders and motivational messages to friends; however, 17% were worried about confidentiality. In observation of participant task completion, 94% were able to successfully confirm visit attendance and 73% could request appointment date change. More participants in 4–6 months group completed tasks correctly compared to 1–3 months group, although not significantly different (78% vs. 66%, p = 0.181). Qualitative results were overwhelmingly positive, but patients did note confusion with transfer reporting and concern that 2wT would not reach patients without mobile phones or with lower literacy.

Conclusion

The 2wT app for early ART retention appears highly usable and acceptable, hopefully creating a solid foundation for lifelong engagement in care. The HCD approach put the local team central in this process, ensuring that both patients’ and Lighthouse’s priorities, policies, and practices were forefront in 2wT optimization, raising the likelihood of 2wT success in other routine program contexts.

Introduction

Poor retention of people living with HIV (PLHIV) on antiretroviral therapy (ART) negatively affects individual clinical outcomes and viral load suppression [1]. In sub-Saharan Africa (SSA), in particular, suboptimal retention results in missed targets for critical UNAIDS 95-95-95 targets by 2030 [2]. A systematic review of ART retention in SSA estimated 36-month retention at 65% [3]. Early retention, within the first 6 months on ART, can be the most challenging, resulting in concentrated treatment disruptions within the first year on ART [4]. Factors influencing retention on ART are complex and dynamic, including individual (e.g., poverty, poor access to care, work or familial obligations), institutional (e.g. healthcare workers (HCW) shortages) and macro-level factors (e.g., COVID-19; global funding priorities) [1, 5–7]. However, once patients miss visits or have treatment interruptions, early retention efforts are most likely to successfully return patients to care [8].

Digital or mobile health (mHealth) interventions may help reduce retention gaps [9]. One-way “blast” educational messages or reminders (pre-defined, no response) modesty improve retention and adherence [10, 11]. Adding interactive components, such as two-way texting, fosters direct patient to HCW communication and appears more effective at increasing engagement in care [12–14]. Still, many retention apps fail [15–18]. Across much of SSA, formidable technology, infrastructure, and connectivity challenges continue to stymie mHealth [19, 20]. Additionally, potentially increased workload or required levels of digital literacy among HCWs may limit adoption and potential scale-up [21–23]. Policy, governance structures, and changing standards present additional challenges that require additional stakeholder involvement from inception [24]. Lastly, the dearth of rigorous mHealth intervention evaluation reduces the evidence base to inform improvements [25]. Nonetheless, mHealth shows immense promise to support patients, providers, and programs. Digital health innovations must respond with clearer evidence of success or failure on which to build [26].

To accurately reflect the value and fit of innovation to a specific programmatic need, mHealth programs should prioritize assessment of patient usability and acceptability. According to the ISO/IEC 9126–1, usability involves understandability, learnability, operability and attractiveness—or how an intended user correctly or incorrectly interacts with a digital innovation [27]. Usability is often measured quantitatively. Acceptability is more focused on individual beliefs and practices or their attitude towards using the App and may be assessed qualitatively by asking users how they perceive the App to meet their needs and preferences [28]. Usability and acceptability are both essential to designing effective mHealth interventions that may scale successfully [29]. To further amplify diverse stakeholder engagement and to ensure that the App matches diverse user demands, incorporation of elements of human-centered design (HCD)—such as iterative user feedback—can also benefit mHealth usability and acceptability [30–33].

In 2019, University of Washington’s International Training and Education Center for Health (I-TECH) and technology partner, Medic, pioneered a two-way, text-based (2wT) approach for direct provider-to-patient communication for a randomized control trial (RCT) leveraging the open-source Community Health Toolkit (CHT) (https://communityhealthtoolkit.org/) and an HCD approach. The rigorous RCT demonstrated that interactive, hybrid, 2wT between male circumcision (MC) providers and patients during the 14-day post-operative period was as safe as in-person follow-up while significantly reducing provider workload [34]. Patients and providers rated the 2wT system highly for usability and acceptability [35], and 2wT lowered costs while improving post-operative care [36]. 2wT for MC success was likely due to adherence to many of the core characteristics of mHealth excellence, including prioritizing patient accessibility and acceptance, low technology costs, locally-led adaptation, strong stakeholder collaboration, and government partnership for sustained impact [37, 38]. 2wT for MC adapted successfully from RCT to routine settings [39], reaching over 22,000 men in post-MC care by mid-2022.

Building on the strength of the open-source, CHT-based 2wT and its HCD approach, I-TECH and Lighthouse Trust, the largest public provider of ART in Lilongwe, Malawi, partnered with Medic to adapt the 2wT system to improve early ART retention. Lighthouse Trust is a World Health Organization (WHO) recognized centre of excellence, a public Ministry of Health (MoH) provider, and a leading organization in Malawi’s HIV/AIDS response [40]. Lighthouse has five centers of excellence across the country, including the Martin Preuss Clinic (MPC). MPC is the largest facility with over 25,000 patients on ART. All diagnosed HIV-positive individuals who report to clinic reception are registered in the point of care (POC) electronic medical record system (EMRS). Since 2006, the Lighthouse Trust implemented a Back-to-Care (B2C) program to increase long-term ART retention. The B2C program identifies and traces patients who miss clinic appointment by at least 14 days and supports them to continue ART. Although B2C retention efforts at Lighthouse are robust [41, 42] human and financial resource constraints stretch scare resources, leaving B2C service gaps. In public ART clinics in Malawi like Lighthouse, digital health interventions like 2wT could improve early retention while simultaneously reducing patient tracing workload and costs.

Therefore, in 2021, Lighthouse, I-TECH, and Medic developed a 2wT-based intervention to improve ART retention and initiated a quasi-experimental study to determine 12-month retention impact by comparing 500 2wT participants to a historical cohort of 500 new ART initiates at Lighthouse’s MPC Clinic in Lilongwe, Malawi. In this process-oriented paper, we describe the HCD process for the 2wT for early ART retention app. We present mixed-methods results on patient usability and acceptability with 100 early participants, 50 at 3-months and 50 at 6-months post 2wT enrollment, and report observations of participants interacting with the system. We hypothesized that using an HCD approach and intensive local participation in 2wT optimization would result in high usability and acceptability of 2wT from the patient perspective.

Methods

Study setting

At Lighthouse, patients are initiated on ART following the test and treat strategy, which includes immediate ART initiation for all patients tested HIV positive regardless of clinical or immunological stage [43]. All new initiates receive augmented adherence and HIV disclosure counselling as part of ART initiation. New ART patient visits are scheduled monthly during the first six months on ART and then every three- or six-months if the patient is stable and adherent to ART. Ideally, the B2C program identifies and traces patients who miss clinic appointment by at least 14 days; however, delays in tracing patients are not uncommon, and treatment interruptions may continue beyond 2 weeks. At 60 days, patients failing to return are considered lost to follow-up (LTFU). All ART care, including B2C, is provided free to patients as mandated by MoH.

Study population and recruitment

In the overall, parent quasi-experimental, pre-post study of retention outcomes, eligibility criteria for opt-in 2wT enrolment are: 1) Patients who initiated ART within 6 months; 2) ≥18 years; 3) possession of own phone at enrolment; 4) willing to receive/send SMS; 5) informed consent; and 6) receives confirmed 2wT enrolment text to verify enrolment number. Those without cell phones and who did not wish to participate were excluded from the study and received routine B2C retention support.

Information about 2wT was disseminated during routine patient care as part of ART initiation counseling. New ART initiates who completed B2C locator forms and had mobile phones with them were requested to enroll in 2wT study. Interested patients were then referred to the 2wT site study coordinator for informed consent and enrollment. Those who successfully received a confirmation text were enrolled. Participants were given a thorough overview of the SMS system, including required replies and instructions on how to confirm a clinic appointment, report a transfer out, or reschedule a clinic visit. A total of 500 participants, including 50 from a 2wT system pilot, were enrolled in 2wT group and 500 in the retrospective comparison group for the parent study.

The 100 2wT participants for the usability and acceptability assessment were recruited between August 2021 and January 2022 and selected via a purposeful, strategic sampling approach, enrolling the first 50 participants to reach 6- and 3-months, post 2wT launch, without overlap participants between groups.

2wT system development

Technology background.

Since 2008, Medic equipped more than 41,000 health workers to support more than 80 million care activities across 15 countries [44]. Medic is the steward of the CHT open source project that currently supports almost 60 apps across 23 unique workflows [44, 45], including 2wT-based apps in Malawi, South Africa, and Zimbabwe. Apps built using the CHT work on basic phones, smartphones, tablets, and computers with or without internet connectivity, in any language [46] and are tailored to meet key digital health characteristics recommended by the WHO [33]. The core areas of functionality used by the Malawi 2wT system include peer-to-peer and automated interactive messaging, task management features (unread message; message not delivered), longitudinal patient records, routine syncing, and reports for routine monitoring (e.g. patient response rates, referrals, etc.).

2wT human-centered design.

The Medic-led HCD approach for 2wT for early ART retention [30] focused on patient and provider users’ needs, working in collaborative, multidisciplinary teams (Fig 1). Before 2wT system adaptation, system developers, HCWs, monitoring and evaluation (M&E) teams, B2C staff and patients discussed and specified the context use. A brief formative research was conducted to inform adaptation and implementation. Then, a pilot of 50 ART patients solicited end-user experience in message context, optimal message delivery timing and language preferences (Chichewa or English) with the aim to launch a patient-centered, usable, and acceptable app. Formative message research and pilot testing results also ensured participants’ privacy and confidentiality were protected, making changes to messaging protocols to improve patient protections. Prior to updating the system messages, text messages were reviewed by expert patients (experienced ART patient mentors) for clarity and understandability. After system deployment, weekly technical feedback meetings were conducted to provide a platform to incorporate the voices and experiences of patient users in further refinement processes.

[Figure omitted. See PDF.]

Medic’s HCD approach was utilized for designing, developing and optimizing 2wT for MC post-operative care [30, 47]. Republished from [47] under a CC BY license, with permission from B. Wasunna, original copyright, Medic 2020.

System components and functionalities.

The 2wT system (Fig 2) is a hybrid automated and manual texting interaction between patients and healthcare workers. In the 2wT system, weekly motivational messages are sent without a response request. Individually tailored appointment reminders are sent in anticipation of a visit with a response requested. Multiple responses and free text are accepted at any time. Messages are sent either in English or Chichewa according to patients’ preferences. Patients can send a message with any content at any time in any language. Generally, the 2wT officer responds to any SMS during routine clinic hours within 1–2 days of any message. Patients can opt out of motivational and/or visit reminder messages at any time.

[Figure omitted. See PDF.]

This hybrid automated-interactive 2wT flow diagram illustrates typical pathways of client interaction with the system and 2wT officers.

2wT intervention: Weekly motivational messages.

The 2wT system broadcasts weekly motivation messages with a new health-related message sent each week; example are shown in Table 1. The messages are sent from the same Lighthouse aggregator number and contain no information related to HIV or AIDS. Messages repeat after approximately 6 months. For weekly motivational messages, there is no response required nor requested from patients. Patients may respond if they choose.

[Figure omitted. See PDF.]

2wT intervention: Visit reminders and responses.

The 2wT system sends individualized clinic visit reminders with a response requested. Patients are asked to respond to the visit reminder messages with a single number: 1 = yes if confirming clinic attendance or 0 = no if they will not attend (Fig 2). A "yes” triggers, "See you soon!" and stops visit reminders for that specific visit. A "no" triggers automated, scripted texts to determine if someone has transferred or wants to change their visit date. The system also allows for free text interaction between 2wT staff and the patient to determine next steps. Patients can request an appointment date change, report transfers, or request a call. Patients receive automated missed visit alerts on days 2, 5, and 11 post appointment date when a clinic appointment is missed. Motivational and reminder visit messages are muted if or when specific ART outcomes are ascertained (stopping ART, transferring to other clinic and death). A 2wT officer prompt for B2C tracing is automatically triggered by the system when a patient misses a clinic appointment by 14 days.

Data collection and management

A brief, semi-structured, interviewer-administered questionnaire (S1 File) contained 16 closed-ended questions to assess 2wT usefulness, acceptability, perceived benefits, and barriers. In open-ended questions, patients were asked for feedback on the intervention, express concerns and suggest improvements.

Participants were also observed completing key 2wT tasks to assess the effectiveness of 2wT education at enrollment and to help improve participant’s adherence to 2wT prompts. Participants were shown four pictures of 2wT visit reminder messages and asked to indicate, using words or touching their phone, how they would: (a) confirm visit attendance, (b) transfer out to another clinic, (c) change appointment dates; (d) ask for help with something else (Fig 3). Results from assessment methods were combined to understand 2wT usability and acceptability from patient users.

[Figure omitted. See PDF.]

Answer/education sheet showing clients how to respond to visit reminders, transfer reporting, visit date change requests, and requests for additional help.

Statistical analysis

Quantitative data analysis was conducted using Stata software version 17.0 (S2 File). For questionnaires on acceptability, frequencies were used to explore closed-ended or partially open-ended responses (S3 File Short answers). Atlas.ti 8 was used for qualitative data from open-ended short answers. Qualitative data were entered and coded by one coder using a mix of deductive and inductive approaches guided by the questionnaire and thematic content. Initial thematic content was shared with the study team for feedback and revisions incorporated into final results.

Ethics

The parent study, including the usability and acceptability components, was reviewed and approved by the University of Washington and the Malawi National Health Sciences Research Committee. All participants provided written informed consent.

Results

Demographic characteristics

One hundred 2wT participants were included in this sub-study (Table 2), including 28 from the pilot period. On average, participants were age 37.1 (SD 9.4) years; almost 40% of participants were under 35 years old. The majority (≥67%) of the participants started ART with WHO stages 1 or 2.

[Figure omitted. See PDF.]

2wT app usability

Most participants were satisfied with visit reminders (95%) and the motivational messages (90%) (Table 3). Over 88% would recommend both. However, 9% did not understand the weekly motivational messages schedule, while over 25% expressed challenges responding correctly to reminders. There were also some confidentiality concerns: 17% were worried about breaches.

[Figure omitted. See PDF.]

2wT task completion

All 100 participants completed four tasks designed to test their ability to respond to 2wT messages. Overall, 72% answered correctly to interactive message task: 94% visit attendance confirmation, 72% informing the facility of transferring to another clinic, 73% changing clinic appointment dates task 1, and 99% changing clinic appointment date task 2 (Table 4). Overall, more participants in the 4–6 months group completed the task correctly compared to those in the 1–3 months group, although not significantly different (78% vs. 66%, p = 0.181). Both groups had difficulties completing the transfer-out and appointment dates tasks.

[Figure omitted. See PDF.]

Qualitative responses on acceptability and usability

Of 100 respondents, 95 provided responses to the open-ended questions (S3 File). Most respondents provided positive feedback and support for the 2wT intervention, asking the team to "continue sending us messages, most people forget their clinic appointment." The vast majority, noted their appreciation for the visit reminders, like this patient:

"This system is good and continue offering us this service. We got reminded when we forget the dates. I came on 26 January for consultation, but I didn’t get drugs then when my days were coming close, I received the reminder message. I thought in my mind that I had already received the meds. Thank you for reminding me."

Participants overwhelmingly noted that they felt encouraged and motivated by the messages, appearing to perceive the messages as coming from a member of the Lighthouse support team. Respondents, like this person, asked the team to, "continue sending messages as they are encouraging us to stay healthy. It reminds me and motivates me." Another participant noted, "when I see the messages, I do have the conscious to take the medicine." Patients felt cared for and supported by the Lighthouse team, noting feelings of comfort such as, "through these interactions, we are not afraid as we know you will help us," and "I am very thankful when I receive these messages."

Patients also liked the weekly SMS health-related message, as one patient reporting: "I am thankful because the messages remind me of my clinic appointment and encourages me to live a healthy life." 2wT was also received positively as a good alternative to reminder phone calls. One participant expressed that "SMS visit reminder is better than through phone call because colleagues ask us why someone is calling me."

There were several themes that suggest the need for improvement. A few patients noted confusion with how to stop messages while others noted delays in HCW responses to their visit change or support requests. One patient also requested changes to support lower literacy patients, remarking that, "the messages are good for people who know how to read. You should also prepare something for those who cannot read." Lastly, a few patients receive missed visit reminders when they attended their visit on time, leaving them frustrated. "Stop sending messages reminding us that we have missed the visit when we already reported."

Not all responses were in support of 2wT. A few patients noted that even without explicit mention of ART nor the clinic name, they still had concerns about disclosure. One patient noted that they were "always afraid that someone might see the messages as my phone is always with neighbors. Someone who knows the number XXXX might realize that this message is from Lighthouse and might compromise with my privacy."

Discussion

Likely as a result of the HCD-based adaption of the 2wT model, we found 2wT to be highly usable and acceptable as an early retention intervention for patients at MPC’s routine ART program. Although some patients, including those who are older or illiterate, may prefer reminder calls [14], these results indicate that the vast majority appreciated both the visit and motivational messages and would recommend 2wT for their friends. However, a quarter of respondents had trouble responding correctly to messages and there were some who expressed concerns with confidentiality, namely people seeing their messages. Compared to those in the 1–3 months in 2wT group, slightly more participants in the 4–6 month group correctly completed the tasks, showing that patient understanding and confidence likely increase over time. Although confirming visit attendance was considered simple, almost 30% found it difficult to report a transfer out or reschedule a clinic visit respectively. These findings suggest several strengths and areas of improvement for 2wT.

Iterative, highly-participatory, HCD was central to App optimization. Routine B2C retention staff helped co-design, develop, test, and optimize 2wT through a series of formative input sessions that also included ART patients, monitoring and evaluation staff. Cycles of optimization strengthened app specifics including: message frequency; motivational message content; timing of delivery; and wording of the automated messages. Pre-testing was also critical: the pilot test conducted among 50 participants prior to the roll-out helped to identify system weaknesses, coding flaws, and bugs which were rectified before study implementation. The hybrid approach, blending automated motivation messages and individually tailored visit reminders with access to direct, interactive, healthcare worker-to-patient messaging, put a human face to the intervention. Although SMS was the primary form of communication, phone calls were made at patient request. This flexibility to respond to patients via SMS or phone also demonstrates prioritization of patient preferences. Patients believed and were assured that a member of the Lighthouse team was supporting them in the manner they preferred, likely contributing to 2wT satisfaction.

Multi-method formative assessment of usability also added rich data on which to improve the system for patient users. Quantitative, observational, and qualitative data each provided key information to ensure that 2wT was both easy to use and highly useful to new ART patients, resulting in a system that helped patients attend visits and stay motivated in care. Other research also used multiple methods to help ensure adaptation and optimization for the local context. In South Africa, an app for HIV self-testing showed that nearly all (98.7%) found the app easy to use and reported that 89.0% of participants could follow all the steps on the app without error [48]. However, usability does not always indicate usefulness. In another South African app for HIV self-testing, high usability (87.8%) did not lead to participants to log onto the app: only 22% self- reported results [49]. In Malawi, assessment of two electronic community case management Apps to support decision making for childhood illnesses found significant differences between the two apps in ease of use [50].

There are several areas for improvement in the 2wT system. First, participants received training on responding to the 2wT system messages at enrolment only, which likely contributed to observed differences in usability over time. Practice and repetition, complemented by refresher patient education (in-person or via posters) during subsequent clinic visits may be recommended as patients tend to understand the system over time. Second, similar to other studies [51–53], confidentiality concerns or fear of disclosure of HIV status using SMS for ART reminders, were reported by some participants. Several protections were implemented to reduce confidentiality concerns. 2wT message co-creation and pre-testing with patients attempted to ensure that both motivational and visit reminder messages were neutral and benign, reducing potential associations with HIV-related care or services. 2wT was also implemented only as opt-in: patients were consented after 2wT educational counseling that included a presentation with example messages. Those who self-reported phone sharing were also excluded. However, this may not have been enough as 17% of the participants reported that they were afraid someone else would see the messages. Additional protections should be explored. Lastly, 2wT is limited only to new ART enrollees and those with mobile phones and is intended to contribute to, and not replace, the routine Back to Care retention activities. However, reducing the literacy level required for participation or considerations of how to include those with shared phones would improve access to this retention support.

Limitations

As with all interviewer-led questionnaires, it is possible that patients were affected by the interviewer, a bias that could have influenced responses in a more positive or negative direction. Usability could change over time. One hundred patients were included in the usability assessment, but de-identified data linked to study-specific IDs resulted in errors linking usability results to patient demographics, reducing the sample size for demographics from 100 to 98. Lastly, SMS response rates, retention outcomes and cost data is forthcoming. It is expected that these results will shed additional light on whether high usability and acceptability will result in effectiveness of the 2wT approach on retention as expected.

Conclusions

2wT for early ART retention at Lighthouse Trust was purposefully designed with patients at the forefront. As a result, it appears that the App was highly usable and acceptable to those who chose to enroll in the retention support intervention, perhaps setting them up for a successful start to lifelong engagement in care. Although 2wT is not for all patients in all settings, the 2wT intervention approach has several strengths that are worthy of consideration for other digital health collaborators working to improve early ART retention. The local team led this process, ensuring that needs, priorities, and practices from Lighthouse, and not from international collaborators, were centered in the decision-making [54]. 2wT was purposefully built using the open-source Community Health Toolkit (CHT), a global digital good. By leveraging an existing set of tools, and adapting from previous 2wT-based evidence [34], delays with App development were reduced while the potential for sustainability increased. Lastly, as 2wT is implemented in a routine, MoH ART services context, the HCD process reflected the realities of routine, and not research, settings, raising the likelihood of longer-term fidelity and potential scale-up of the 2wT intervention in other LMIC programs.

Supporting information

S1 File. Usability survey.

This interviewer-administered survey was employed to assess usability.

https://doi.org/10.1371/journal.pone.0278806.s001

(CSV)

S2 File. Dataset.

Usability dataset in CSV format.

https://doi.org/10.1371/journal.pone.0278806.s002

(CSV)

S3 File. Dataset.

Qualitative short-answer responses from usability survey (English).

https://doi.org/10.1371/journal.pone.0278806.s003

S4 File.

https://doi.org/10.1371/journal.pone.0278806.s004

(DOCX)

Acknowledgments

We would like to acknowledge the study participants, the study team (Kondwani Masiye, Harrison Chirwa, Blessings Wandira, Daniel Mwakanema, Madalitso Chawanje, William Maziya, Isaac Nyirenda), colleagues from medic mobile (Edwin Kagereki, Adinan Alhassan, Mourice Barasa), the MPC clinic, M&E, retention team, and MoH staff at Bwaila for their valued contribution to the study.

Citation: Huwa J, Tweya H, Mureithi M, Kiruthu-Kamamia C, Oni F, Chintedza J, et al. (2023) “It reminds me and motivates me”: Human-centered design and implementation of an interactive, SMS-based digital intervention to improve early retention on antiretroviral therapy: Usability and acceptability among new initiates in a high-volume, public clinic in Malawi. PLoS ONE 18(7): e0278806. https://doi.org/10.1371/journal.pone.0278806

About the Authors:

Jacqueline Huwa

Roles: Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Lighthouse Trust, Lilongwe, Malawi

ORICD: https://orcid.org/0000-0001-8786-2487

Hannock Tweya

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: International Training and Education Center for Health (I-TECH), Lilongwe, Malawi

Maryanne Mureithi

Roles: Investigation, Methodology, Software, Validation, Visualization, Writing – review & editing

Affiliation: Medic mobile, Nairobi, Kenya

Christine Kiruthu-Kamamia

Roles: Supervision, Writing – review & editing

Affiliation: Lighthouse Trust, Lilongwe, Malawi

Femi Oni

Roles: Software, Supervision, Validation, Visualization, Writing – review & editing

Affiliation: Medic mobile, Nairobi, Kenya

Joseph Chintedza

Roles: Investigation, Project administration, Supervision, Writing – review & editing

Affiliation: Lighthouse Trust, Lilongwe, Malawi

Geldert Chiwaya

Roles: Investigation, Project administration, Supervision, Writing – review & editing

Affiliation: Lighthouse Trust, Lilongwe, Malawi

Evelyn Waweru

Roles: Formal analysis

Affiliation: Medic mobile, Nairobi, Kenya

ORICD: https://orcid.org/0000-0002-4599-8973

Aubrey Kudzala

Roles: Project administration, Supervision, Writing – review & editing

Affiliation: Lighthouse Trust, Lilongwe, Malawi

Beatrice Wasunna

Roles: Software, Supervision, Validation, Visualization, Writing – review & editing

Affiliation: Medic mobile, Nairobi, Kenya

Dumisani Ndhlovu

Roles: Software, Supervision, Validation, Writing – review & editing

Affiliation: Lighthouse Trust, Lilongwe, Malawi

Pachawo Bisani

Roles: Project administration, Visualization, Writing – review & editing

Affiliation: Lighthouse Trust, Lilongwe, Malawi

Caryl Feldacker

Roles: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Resources, Supervision, Writing – original draft, Writing – review & editing

Affiliations: Department of Global Health, University of Washington, Seattle, WA, United States of America, International Training and Education Center for Health (I-TECH), Seattle, WA, United States of America

ORICD: https://orcid.org/0000-0002-8152-6754

References

1. UNAIDS. IN DANGER: UNAIDS Global AIDS Update 20222022. https://www.unaids.org/sites/default/files/media_asset/2022-global-aids-update_en.pdf, Search.tb.ask.com [Internet]. Search.tb.ask.com. [cited 2022 Nov 20]. https://search.tb.ask.com/web

2. UNAIDS. Not there yet on viral suppression [Internet]. [cited 2022 Nov 20]. https://www.unaids.org/en/resources/presscentre/featurestories/2019/march/20190304_gow_viral_load

3. Fox MP. Retention of adult patients on antiretroviral therapy in low-and middle-income countries: systematic review and meta-analysis 2008–2013. J Acquir Immune Defic Syndr 1999. 2015;69(1):98. pmid:25942461

4. Haas AD, Zaniewski E, Anderegg N, Ford N, Fox MP, Vinikoor M, et al. Retention and mortality on antiretroviral therapy in sub‐Saharan Africa: collaborative analyses of HIV treatment programmes. J Int AIDS Soc. 2018;21(2):e25084. pmid:29479867

5. Mukumbang FC, Mwale JC, van Wyk B. Conceptualising the factors affecting retention in care of patients on antiretroviral treatment in Kabwe District, Zambia, using the ecological framework. AIDS Res Treat. 2017;2017. pmid:29250442

6. Chamberlin S, Mphande M, Phiri K, Kalande P, Dovel K. How HIV Clients Find Their Way Back to the ART Clinic: A Qualitative Study of Disengagement and Re-engagement with HIV Care in Malawi. AIDS Behav. 2022;26(3):674–85. pmid:34403022

7. Nyoni T, Okumu M. COVID-19-compliant strategies for supporting treatment adherence among people living with HIV in sub-Saharan Africa. AIDS Behav. 2020;24(9):2473–6. pmid:32333204

8. Chammartin F, Zürcher K, Keiser O, Weigel R, Chu K, Kiragga AN, et al. Outcomes of patients lost to follow-up in African antiretroviral therapy programs: individual patient data meta-analysis. Clin Infect Dis. 2018;67(11):1643–52. pmid:29889240

9. Demena BA, Artavia-Mora L, Ouedraogo D, Thiombiano BA, Wagner N. A systematic review of mobile phone interventions (SMS/IVR/calls) to improve adherence and retention to antiretroviral treatment in low-and middle-income countries. AIDS Patient Care STDs. 2020;34(2):59–71. pmid:32049555

10. Linde DS, Korsholm M, Katanga J, Rasch V, Lundh A, Andersen MS. One-way SMS and healthcare outcomes in Africa: systematic review of randomised trials with meta-analysis. PLoS One. 2019;14(6):e0217485. pmid:31170176

11. Jong S, Cuca Y, Thompson LM. Meta-analysis of mobile phone reminders on HIV patients’ retention to care. J Mob Technol Med. 2017;6(1):5. pmid:30197685

12. Ødegård ES, Langbråten LS, Lundh A, Linde DS. Two-way text message interventions and healthcare outcomes in Africa: Systematic review of randomized trials with meta-analyses on appointment attendance and medicine adherence. PloS One. 2022;17(4):e0266717. pmid:35421134

13. Mills EJ, Lester R, Thorlund K, Lorenzi M, Muldoon K, Kanters S, et al. Interventions to promote adherence to antiretroviral therapy in Africa: a network meta-analysis. Lancet HIV. 2014;1(3):e104–11. pmid:26424119

14. Amankwaa I, Boateng D, Quansah DY, Akuoko CP, Evans C. Effectiveness of short message services and voice call interventions for antiretroviral therapy adherence and other outcomes: A systematic review and meta-analysis. PLoS One. 2018;13(9):e0204091. pmid:30240417

15. Linnemayr S, Huang H, Luoto J, Kambugu A, Thirumurthy H, Haberer JE, et al. Text messaging for improving antiretroviral therapy adherence: no effects after 1 year in a randomized controlled trial among adolescents and young adults. Am J Public Health. 2017;107(12):1944–50. pmid:29048966

16. Mbuagbaw L, Thabane L, Ongolo-Zogo P, Lester RT, Mills EJ, Smieja M, et al. The Cameroon Mobile Phone SMS (CAMPS) trial: a randomized trial of text messaging versus usual care for adherence to antiretroviral therapy. PloS One. 2012;7(12):e46909. pmid:23236345

17. Kliner M, Knight A, Mamvura C, Wright J, Walley J. Using no-cost mobile phone reminders to improve attendance for HIV test results: a pilot study in rural Swaziland. Infect Dis Poverty. 2013;2(1):1–7.

18. Gross R, Ritz J, Hughes MD, Salata R, Mugyenyi P, Hogg E, et al. Two-way mobile phone intervention compared with standard-of-care adherence support after second-line antiretroviral therapy failure: a multinational, randomised controlled trial. Lancet Digit Health. 2019;1(1):e26–34. pmid:31528850

19. Kruse C, Betancourt J, Ortiz S, Luna SMV, Bamrah IK, Segovia N. Barriers to the use of mobile health in improving health outcomes in developing countries: systematic review. J Med Internet Res. 2019;21(10):e13263. pmid:31593543

20. Karlyn A, Odindo S, Onyango R, Mbindyo C, Mberi T, Too G, et al. Testing mHealth solutions at the last mile: insights from a study of technology-assisted community health referrals in rural Kenya. Mhealth. 2020;6. pmid:33437839

21. Krebs P, Duncan DT. Health app use among US mobile phone owners: a national survey. JMIR MHealth UHealth. 2015;3(4):e4924. pmid:26537656

22. Gagnon MP, Ngangue P, Payne-Gagnon J, Desmartis M. m-Health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc. 2016;23(1):212–20. pmid:26078410

23. Qudah B, Luetsch K. The influence of mobile health applications on patient-healthcare provider relationships: a systematic, narrative review. Patient Educ Couns. 2019;102(6):1080–9. pmid:30745178

24. Desveaux I. overcoming policy-level barriers to the implementation of digital health innovation: Qualitative study. J Med Internet Res. (21).

25. Guo C, Ashrafian H, Ghafur S, Fontana G, Gardner C, Prime M. Challenges for the evaluation of digital health solutions—A call for innovative evidence generation approaches. NPJ Digit Med. 2020;3(1):1–14. pmid:32904379

26. Shaw J, Agarwal P, Desveaux L, Palma DC, Stamenova V, Jamieson T, et al. Beyond “implementation”: digital health innovation and service design. NPJ Digit Med. 2018;1(1):1–5. pmid:31304327

27. Bevan N, Carter J, Harker S. ISO 9241–11 revised: What have we learnt about usability since 1998? In Springer; 2015. p. 143–51.

28. Nadal C, Sas C, Doherty G. Technology acceptance in mobile health: scoping review of definitions, models, and measurement. J Med Internet Res. 2020;22(7):e17256. pmid:32628122

29. Goldberg L, Lide B, Lowry S, Massett HA, O’Connell T, Preece J, et al. Usability and accessibility in consumer health informatics: current trends and future challenges. Am J Prev Med. 2011;40(5):S187–97. pmid:21521594

30. Holeman I, Kane D. Human-centered design for global health equity. Inf Technol Dev. 2020;26(3):477–505.

31. Göttgens I, Oertelt-Prigione S. The application of human-centered design approaches in health research and innovation: a narrative review of current practices. JMIR MHealth UHealth. 2021;9(12):e28102. pmid:34874893

32. Johnson T, Das S, Tyler N. Design for health: human-centered design looks to the future. Glob Health Sci Pract. 2021;9(Supplement 2):S190–4. pmid:34845042

33. World Health Organization. WHO guideline: recommendations on digital interventions for health system strengthening: web supplement 2: summary of findings and GRADE tables [Internet]. World Health Organization; 2019. https://apps.who.int/iris/bitstream/handle/10665/324998/WHO-RHR-19.7-eng.pdf.

34. Feldacker C, Murenje V, Holeman I, Xaba S, Makunike-Chikwinya B, Korir M, et al. Reducing Provider Workload While Preserving Patient Safety: A Randomized Control Trial Using 2-Way Texting for Postoperative Follow-up in Zimbabwe’s Voluntary Medical Male Circumcision Program. J Acquir Immune Defic Syndr 1999. 2020;83(1):16. pmid:31809358

35. Feldacker C, Holeman I, Murenje V, Xaba S, Korir M, Wambua B, et al. Usability and acceptability of a two-way texting intervention for post-operative follow-up for voluntary medical male circumcision in Zimbabwe. PLOS ONE. 2020 Jun 16;15(6):e0233234. pmid:32544161

36. Babigumira JB, Barnhart S, Mendelsohn JM, Murenje V, Tshimanga M, Mauhy C, et al. Cost-effectiveness analysis of two-way texting for post-operative follow-up in Zimbabwe’s voluntary medical male circumcision program. PLOS ONE. 2020 Sep 30;15(9):e0239915. pmid:32997710

37. Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health. 2014 Feb 21;14(1):188.

38. Bauer AM, Hodsdon S, Bechtel JM, Fortney JC. Applying the Principles for Digital Development: Case Study of a Smartphone App to Support Collaborative Care for Rural Patients With Posttraumatic Stress Disorder or Bipolar Disorder. J Med Internet Res. 2018 Jun 6;20(6):e10048. pmid:29875085

39. Marongwe P, Wasunna B, Gavera J, Murenje V, Gwenzi F, Hove J, et al. Transitioning a digital health innovation from research to routine practice: Two-way texting for male circumcision follow-up in Zimbabwe. PLOS Digit Health. 2022 Jun 15;1(6):e0000066. pmid:36812548

40. Phiri S, Neuhann F, Glaser N, Gass T, Chaweza T, Tweya H, et al. The path from a volunteer initiative to an established institution: evaluating 15 years of the development and contribution of the Lighthouse trust to the Malawian HIV response. BMC Health Serv Res. 2017 Aug 9;17(1):548.

41. Tweya H, Gareta D, Chagwera F, Ben-Smith A, Mwenyemasi J, Chiputula F, et al. Early active follow-up of patients on antiretroviral therapy (ART) who are lost to follow-up: the ‘Back-to-Care’ project in Lilongwe, Malawi. Trop Med Int Health. 2010;15(s1):82–9. pmid:20586965

42. Tweya H, Oboho IK, Gugsa ST, Phiri S, Rambiki E, Banda R, et al. Loss to follow-up before and after initiation of antiretroviral therapy in HIV facilities in Lilongwe, Malawi. PLOS ONE. 2018 Jan 26;13(1):e0188488. pmid:29373574

43. Malawi Clinical HIV Guidelines 2018 (1).pdf [Internet]. [cited 2022 Nov 20]. https://differentiatedservicedelivery.org/Portals/0/adam/Content/yb4xSSLvE0SW98_z7wTm_w/File/Malawi%20Clinical%20HIV%20Guidelines%202018%20(1).pdf

44. Medic. 2021 Annual Report [Internet]. Medic. 2022 [cited 2022 Nov 20]. https://medic.org/2021-annual-report/

45. Medic. Community Health Toolkit [Internet]. Community Health Toolkit. [cited 2022 Nov 20]. https://communityhealthtoolkit.org

46. Jumreornvong O. New Weapon In The War Against AIDS: Your Mobile Phone. Intersect Stanf J Sci Technol Soc [Internet]. 2014 Apr 26 [cited 2022 Nov 20];7(1). Available from: https://ojs.stanford.edu/ojs/index.php/intersect/article/view/599

47. Wasunna B, Holeman I. Digital Health Interventions for Community Health Worker Training, Ongoing Education, and Supportive Supervision. Training for Community Health: Bridging the Global Health Care Gap. 2021;67.

48. Gous N, Fischer AE, Rhagnath N, Phatsoane M, Majam M, Lalla-Edward ST. Evaluation of a mobile application to support HIV self-testing in Johannesburg, South Africa. South Afr J HIV Med. 2020;21(1):1–7. pmid:32670629

49. Fischer AE, Phatsoane M, Majam M, Shankland L, Abrahams M, Rhagnath N, et al. Uptake of the Ithaka mobile application in Johannesburg, South Africa, for human immunodeficiency virus self-testing result reporting. South Afr J HIV Med. 2021;22(1):1197. pmid:33824737

50. Chirambo GB, Muula AS, Thompson M, Hardy VE, Heavin C, O’Connor Y, et al. End-user perspectives of two mHealth decision support tools: Electronic Community Case Management in Northern Malawi. Int J Med Inf. 2021;145:104323. pmid:33232917

51. Crankshaw T, Corless IB, Giddy J, Nicholas PK, Eichbaum Q, Butler LM. Exploring the patterns of use and the feasibility of using cellular phones for clinic appointment reminders and adherence messages in an antiretroviral treatment clinic, Durban, South Africa. AIDS Patient Care STDs. 2010;24(11):729–34. pmid:21039181

52. van der Kop ML, Muhula S, Nagide PI, Thabane L, Gelmon L, Awiti PO, et al. Effect of an interactive text-messaging service on patient retention during the first year of HIV care in Kenya (WelTel Retain): an open-label, randomised parallel-group study. Lancet Public Health. 2018;3(3):e143–52. pmid:29361433

53. Ronen K, Unger JA, Drake AL, Perrier T, Akinyi P, Osborn L, et al. SMS messaging to improve ART adherence: perspectives of pregnant HIV-infected women in Kenya on HIV-related message content. AIDS Care. 2018;30(4):500–5. pmid:29254362

54. Bartels SM, Haider S, Williams CR, Mazumder Y, Ibisomi L, Alonge O, et al. Diversifying Implementation Science: A Global Perspective. Glob Health Sci Pract [Internet]. 2022 Aug 30 [cited 2022 Nov 20];10(4). Available from: https://www.ghspjournal.org/content/10/4/e2100757 pmid:36041849