Impact of a Daily SMS Medication Reminder System on Tuberculosis Treatment Outcomes: A Randomized Controlled Trial

Abstract

Importance

The rapid uptake of mobile phones in low and middle-income countries over the past decade has provided public health programs unprecedented access to patients. While programs have used text messages to improve medication adherence, there have been no high-powered trials evaluating their impact on tuberculosis treatment outcomes.

Objective

To measure the impact of Zindagi SMS, a two-way SMS reminder system, on treatment success of people with drug-sensitive tuberculosis.

Design

We conducted a two-arm, parallel design, effectiveness randomized controlled trial in Karachi, Pakistan. Individual participants were randomized to either Zindagi SMS or the control group. Zindagi SMS sent daily SMS reminders to participants and asked them to respond through SMS or missed (unbilled) calls after taking their medication. Non-respondents were sent up to three reminders a day.

Setting

Public and private sector tuberculosis clinics in Karachi, Pakistan.

Participants

Newly-diagnosed patients with smear or bacteriologically positive pulmonary tuberculosis who were on treatment for less than two weeks; 15 years of age or older; reported having access to a mobile phone; and intended to live in Karachi throughout treatment were eligible to participate. We enrolled 2,207 participants, with 1,110 randomized to Zindagi SMS and 1,097 to the control group.

Main Outcome

The primary outcome was clinically recorded treatment success based upon intention-to-treat.

Results

We found no significant difference between the Zindagi SMS or control groups for treatment success (719 or 83% vs. 903 or 83%, respectively, p = 0·782). There was no significant program effect on self-reported medication adherence reported during unannounced visits during treatment.

Conclusion

In this large-scale randomized controlled effectiveness trial of SMS medication reminders for tuberculosis treatment, we found no significant impact.

Trial Registration

The trial was registered with ClinicalTrials.gov, NCT01690754.

About the Authors:

Shama Mohammed

* E-mail: [email protected]

Affiliation: Interactive Research and Development, Karachi, Pakistan

Rachel Glennerster

Affiliation: Abdul Latif Jameel Poverty Action Lab, Massachusetts Institute of Technology, Cambridge, MA, United States of America

Aamir J. Khan

Affiliation: Interactive Research and Development, Karachi, Pakistan

Introduction

Tuberculosis is the second-leading cause of death from infectious diseases globally, with nine million people infected and 1.5 million deaths in 2013 [1]. Treatment for drug-sensitive tuberculosis lasts six to eight months and can result in difficult side effects. Failure to adhere to treatment can result in continued transmission, the development of multidrug-resistant tuberculosis, or death. The World Health Organization recommends directly observed therapy to promote adherence, where a pre-assigned treatment supporter watches each patient take his/her daily medication. However, evidence for the effectiveness of this method is inconclusive [2].

The rapid uptake of mobile phones in low- and middle-income countries has provided public-health programs unprecedented access to patients. Mobile phone-based interventions to improve medication adherence have been adopted for many diseases, with mixed results[3–6]. Most trials are inadequately designed, insufficiently powered, or are restricted to high-income countries [3–6]. The most rigorous trials evaluating mobile phone-use for treatment compliance in developing countries exist for adherence to antiretroviral therapy (ART) for people living with Human Immunodeficiency Virus (HIV). Meta-analysis that combines two positive trials in Kenya [7,8] and one null result from Cameroon [9] found an overall positive impact of weekly Short Message Service (SMS, or text message) reminders [10–12]. A more recent study of interactive automated voice reminders and pictorial messages in India, found no impact on adherence to ART [13].

While there is considerable interest in the potential of mobile phone-based interventions to improve tuberculosis treatment adherence, rigorous evidence on their impact is limited [14–17]. Recently, a limited cluster randomized trial in China found that, while medication monitor reminders led to improved drug compliance for tuberculosis patients, daily two-way SMS reminders did not [18]. Our effectiveness trial gauged the impact of a two-way interactive SMS medication reminder system (Zindagi SMS) on the treatment success of people with drug-sensitive tuberculosis.

Methods

Study Design and Participants

Participants were recruited into the randomized controlled trial through a large tertiary center (the Indus Hospital), nine public facilities, and a network of private General Practitioner (GP) clinics and private laboratories in Karachi.

To be eligible, participants had to be newly diagnosed with smear or bacteriologically positive pulmonary tuberculosis; 15 years of age or older; report having access to a mobile phone; and intend to live in Karachi throughout treatment. To allow Zindagi SMS to help establish habits early, participants had to be on treatment for less than two weeks. To minimize spillovers, patients with another household member in the study were ineligible. Enrolment continued until the predetermined sample size was met.

Eligible participants were consented using an oral consent form, a copy of which was given to the participant. Oral consent was solicited instead of written consent, as our sample was a low-literate population. The trial and oral consent procedure was approved by the research ethics boards at Interactive Research and Development (IRD) in Karachi, Pakistan and the Massachusetts Institute of Technology (MIT) in Cambridge, USA. The trial was registered with ClinicalTrials.gov, NCT01690754.

Randomization and Masking

Once a patient consented to participate, a study representative entered identifying information on a mobile phone-based enrolment form. Individual participants were randomized to either the Zindagi SMS or control groups, using predetermined list on the study server that was generated using simple randomization. The research team was blinded to the allocation sequence generated. If mobile data connectivity was interrupted (10% of enrolments), the study representative called their supervisor, who entered the identifying information into Microsoft Excel and generated the group assignment using the randomization function. In both cases field staff who interacted directly with patients had no ability to influence the randomized allocation. The randomization status of individual participants was not shared with treating clinics by the research team.

Procedures

All study participants received the standard of care provided by their clinic. Initially, participants received NTP’s recommended eight-month treatment regimen. However, NTP guidelines changed in the second quarter of 2012 to a six-month regimen.

Zindagi SMS used two-way reminders to encourage patients to actively engage with reminders, rather than passively read and potentially ignore them. It enabled the study team to identify non-responsive patients for phone calls to encourage participation. Zindagi SMS sent enrolled patients daily reminders at a time of their choosing. Messages were in Urdu using English script and included a daily motivational message followed by a reminder to respond via SMS or, after September 2011, a missed (unbilled) call to indicate they had taken their medication. Fourteen messages were randomized and sent to participants. Based on feedback from our pilot, tuberculosis was not mentioned in the messages due to stigma [19]. For example, one message said, “Your health is in your hands. Take your medication and remember to respond by SMS or a missed call.” SMS responses were not verified for content. Participants were offered PKR 60 (USD 0.60) per month to cover the costs of responding. Initially, the participants were asked to pick up reimbursements at their clinic, but from October 2013, reimbursements were transferred directly to participants’ phones.

Once Zindagi SMS received a response or a missed call, a confirmatory SMS message was sent to the respondent. If the patient did not respond within two hours, a second reminder was sent. A third and final reminder for the day was sent after two additional hours of non-responsiveness. Members of our study team phoned participants who did not respond for seven days.

Participants were interviewed at their household for a more extensive baseline. Ninety percent of completed baseline surveys were conducted within 30 days of enrolment. Once a month, study enumerators attempted surprise visits to participants’ households to conduct a midline survey. Sputum samples, independent from those collected by the treating clinics, were collected for patients at various points in their treatment. Given the difficulties in finding participants at home and that they could not always produce sputum, there is considerable variation in the sputum samples available per patient. An endline survey was conducted after the completion of participants’ treatment period. Participants who were reported as having defaulted or transferred out from treatment were surveyed again between September and November 2014 to record whether they continued their treatment.

The enrolment form was entered on mobile phones, with paper forms as backup. Other surveys were collected on paper and double-entered into a Microsoft Access database.

Outcomes

The primary outcome was programmatically defined treatment success as recorded in clinic registers provided by the NTP, the global standard for assessing tuberculosis programs. Treatment success is defined as the sum of patients clinically reported as cured (i.e. patient whose sputum smear or culture was positive at the beginning of treatment but who was smear- or culture-negative in the last month of treatment and on at least one previous occasion) or treatment completed (i.e. a patient who completed treatment but who does not have a negative sputum smear or culture result in the last month or treatment and on at least one previous occasion). In our secondary analysis, we explored the full range of clinically recorded treatment outcomes (cured, treatment completed, default, died, treatment failure and transferred out). In particular, Zindagi SMS might have increased success by reducing default (i.e. a patient whose treatment was interrupted for two consecutive months or more). The limitation of this outcome is that because registers of NTP-reporting clinics in Pakistan are not linked, patients who are reported as having default or transferred out could have continued treatment elsewhere. As a robustness check we therefore explored adjusted treatment outcomes for treatment success and default, substituting self-reported for clinically recorded outcomes for the participants interviewed.

To explore potential mechanisms of impact (or lack of impact), we also gauged adherence as a secondary outcome by asking participants whether they had taken their medication in the last 24 hours during home visits. While self-reported adherence can be unreliable, there is no reason to believe misreporting is systematically different among those assigned to Zindagi SMS or the control group. During survey visits between February and April 2012, we conducted IsoScreen tests, which detect isoniazid metabolites in urine samples to gauge whether tuberculosis drugs were taken within the past 24 hours. Isoniazid is always included in first-line tuberculosis treatment. We compared IsoScreen results with self-reported adherence on the same visit. While IsoScreen tests were conducted on a non-random sample, they give us an indication of the reliability of self-reported adherence.

Finally, we collected self-reported psychological and physical health measures as secondary outcomes. We used a four-point Likert scale for patients’ difficulty in completing a range of physical tasks and how supported they felt during treatment, a picture of a five-rung ladder with six compartments between rungs for likelihood of being cured, and images of five faces for how healthy they felt. Questions on participants’ ability to complete tasks and self-reported health were asked at baseline, midline, and endline; the likelihood of being cured asked at baseline and midline; and how supported participants felt asked at endline.

Statistical Analysis

We calculated a minimum sample size of 1,094 participants in each study arm, using power of 80%, minimum detectable effect size (MDE) of five percentage points, and treatment success rate in the control group of 75%. An MDE of 5 percentage points was chosen because a smaller impact was considered unlikely to motivate policy change. Data were analyzed using intention-to-treat, i.e. allocation to treatment not actual take up. We used the χ² test for differences in proportions for the analysis of the primary outcome. In comparing sputum samples, we ran ordinary least squares regressions, after controlling for days in the study and regimen type.

We also assessed whether Zindagi SMS had differential impacts on treatment success by gender, indicators of quality of care, and mobile phone access. In analyzing our secondary outcomes, we ran ordinary least squares regressions on each outcome to test for program effects, after controlling for days since enrollment and regimen type. To adjust for multiple hypotheses testing we used the Bonferroni correction and the less conservative Westfall and Young free step-down resampling method [20].

Statistical analysis was conducting using STATA/IC version 12.0. A p-value of <0.05 was considered statistically significant.

Results

We enrolled 2,207 participants into the study between March 18, 2011 and February 25, 2014 (Fig 1). Both groups had similar baseline characteristics (Table 1).

[Figure omitted. See PDF.]

Fig 1. Trial Profile.

https://doi.org/10.1371/journal.pone.0162944.g001

[Figure omitted. See PDF.]

Table 1. Demographic characteristics of participants enrolled in the trial (2011–2014, Karachi, Pakistan).^*

https://doi.org/10.1371/journal.pone.0162944.t001

As an effectiveness trial we sought to replicate implementation conditions, as it would exist at scale. The Zindagi SMS system sent reminders or received responses for 174,284 patient days to participants during their estimated treatment duration (180 days for the six-month regimen or 240 days for the eight-month regimen). With perfect implementation, reminders should have been sent for a total of 220,560 patient days, suggesting the system was successfully implemented for 79% of patient days. Missed reminders were due to system failures, administrative shortfalls, or GPRS outages in the city mobile (14%); participants asking to leave the system or dying (3%); participants opting out of receiving reminders at enrolment (2%); and participants not knowing their phone number at enrolment and failing to share their number subsequently (2%). Of the 1,069 participants who were sent messages, 912 (85%) responded at least once. Of the participants that were on the system throughout their treatment, the mean response rate (calculated as the number of responses received over the number of reminders sent per patient) was 29%, ranging from 0 to 99%. Over the course of treatment, average response rates fell from 48% in the first two weeks to 24% (eight-month regimen) and 20% (six-month regimen) in the last two weeks (See Fig 2).

[Figure omitted. See PDF.]

Fig 2. Response rates over time in treatment.

https://doi.org/10.1371/journal.pone.0162944.g002

There were no significant differences in treatment outcomes between participants in the Zindagi SMS group and the control group (Table 2). As a robustness check, we substituted the clinically recorded outcomes with self-reported outcomes for participants who defaulted or transferred out of treatment that we were able to interview, or whose family members reported the participant had died. Of the 283 participants that were reported as having defaulted or transferred out, we interviewed 130 (46%); 49 participants (17%) had died; 22 (8%) refused to be interviewed; and we were unable to locate 82 (29%). Self-reported outcomes were categorized using the criteria in Fig 3. When we adjusted treatment outcomes to reflect the self-reported outcomes, the default rate reduced in both groups but there were still no significant differences in outcomes between the two groups (Table 2). We also examined independent sputum samples between the groups. We collected 1191 sputum samples for participants (603 in the Zindagi SMS group and 588 in the control group). Of these, one hundred and ninety samples were excluded because they were primarily saliva, rather than sputum (104 in the Zindagi SMS group and 86 in the control group). There was no statistically significant difference between sputum results for both groups (p = 0.762).

[Figure omitted. See PDF.]

Fig 3. Criteria for determining self-reported outcomes.

https://doi.org/10.1371/journal.pone.0162944.g003

[Figure omitted. See PDF.]

Table 2. Clinically-recorded treatment success between Zindagi SMS and control groups (2011–2014, Karachi, Pakistan).

https://doi.org/10.1371/journal.pone.0162944.t002

There was no significant program effect in subgroups for gender, any of the indicators relating to quality of care, or access to a mobile phone after adjusting for multiple subgroups using the Bonferroni correction (adjusted p-value of 0.003 for 17 subgroups) or the Westfall and Young free step-down resampling correction (Table 3). We also created a single index of quality of care from relevant variables and another for mobile phone access to test for subgroup effect but again found no significant treatment effects for subgroups. (S1 Appendix)

[Figure omitted. See PDF.]

Table 3. Sub-group analysis using treatment success as the outcome (2011–2014, Karachi, Pakistan).

https://doi.org/10.1371/journal.pone.0162944.t003

We compared the results of 159 IsoScreen tests with self-reported adherence during the same study visit. IsoScreen tests indicated that 17% of those who said they had taken their drugs in the past 24 hours had not, indicating over-reporting. However, these results were not statistically different by treatment arm.

There were no significant program effects in self-reported medication adherence or any of our other secondary outcomes, after adjusting for multiple hypotheses using the Bonferroni correction (p-value of 0.001 for five hypotheses) or the Westfall and Young correction (Table 4).

[Figure omitted. See PDF.]

Table 4. Secondary outcomes between the Zindagi SMS and control groups (2011–2014, Karachi Pakistan).

https://doi.org/10.1371/journal.pone.0162944.t004

Discussion

We found no significant impact of SMS medication reminders on treatment success, other treatment outcomes, self-reported adherence, or self-reported physical and psychological health. There were no statistically significant impacts of reminders on treatment success within a variety of subgroups after adjusting for multiple hypothesis testing.

With an attrition rate of less than 1%, similar across arms, on our primary outcome variable, the results are very robust. The inclusion of participants from public clinics, private GP clinics, laboratories, and a large philanthropic hospital contributes to its external validity.

A limitation of our trial was the lack of an objective adherence measure. Adherence was self-reported, which, IsoScreen tests indicated, is an overestimate of actual adherence. However, even if participation in Zindagi SMS generated increased adherence, not reflected in self-reports, it did not translate into improved treatment outcomes. In addition, IsoScreen tests (on a nonrandom sample) indicated that misreporting was similar in the Zindagi SMS and control groups.

Another potential limitation is that clinics could incorrectly record treatment outcomes to meet expected success rates encouraged by the NTP. However, since clinics were blind to allocation, there is no reason any misreporting was systematically different between treatment arms. Moreover, in our sub-study on participants reported as having defaulted or transferred out, self-reported outcomes were similar in the intervention and control groups. Another potential limitation is that our trial took place in Pakistan, which reports high treatment success rates of 91% in 2012 [1]. Results may differ in countries reporting lower success rates.

Our results were similar to the findings of Liu and colleagues in China that two-way SMS reminders had no impact on medication adherence for tuberculosis patients, though we are able to rule out much smaller effects of SMS reminders [15]. Meta-analysis of studies on HIV and SMS reminders show positive results on of weekly SMS reminders on adherence and clinical outcomes (Finitsis et al 2014 [10], Mbuagbaw et al 2013 [11], Horvath et al 2012) [12]. These reviews draw mainly on three large high quality studies (Finitsis et al also include many small poor quality studies and Horvath et al only includes Lester and colleagues’ [7] and Pop-Eleches and colleagues’[8] trials) and conclude weekly reminders are more effective than daily reminders. The three key studies are Mbuagbaw and colleagues, who test weekly SMS reminders in Cameroon and find no impact [9] Pop-Eleches and colleagues who tested multiple versions of SMS reminders and only found only short messages sent weekly improve adherence [8], and Lester and colleagues find a positive impact of weekly reminders in Kenya [7]. Importantly, in Lester and colleagues, participants also received off-site clinician follow-up if they reported a problem or did not respond to the SMS [7, 21]. Not included in the reviews is Shet and colleagues who test weekly automated telephone reminders in India and find no significant effect [13].

Our study indicates that SMS reminders have no impact on treatment outcomes for patients with drug-sensitive tuberculosis. Participants could operate the system (85% responded at least once), but this did not translate into changed outcomes. We hypothesize that SMS reminders, intended to combat forgetfulness, did not address the underlying factors that contribute to patients leaving treatment. The steady decline of and the low mean of the response rate (29%) suggest that participants tired of daily reminders. It is possible that longer intervals between messages or greater off-site support for remote patients as in the Lester trial might be effective [7, 21–22]. SMS reminders could be more useful for reminding patients of clinic appointments; other forms of appointment reminders (telephone calls, home visits, letters) have helped tuberculosis patients [23]. Finally, SMS reminders could be coupled with financial incentives to motivate patients to stay on course with treatment, especially given the important externalities to the community associated with a patient complying with their treatment regime. Our current study was initially intended to have an arm with incentives for medication adherence, but the technology to link SMS reminders with adherence had manufacturing challenges. Our results suggest pessimism for SMS medication reminders on their own.

In conclusion, we found no impact of SMS reminders on treatment success in tuberculosis patients in the highest-powered such randomized control trial to date.

Supporting Information

[Figure omitted. See PDF.]

S1 Appendix. Additional Sub-group Analysis Using Subgroup Indices.

https://doi.org/10.1371/journal.pone.0162944.s001

(DOCX)

S1 File. CONSORT 2010 Checklist.

https://doi.org/10.1371/journal.pone.0162944.s002

(DOC)

S2 File. Evaluating the Impact of Zindagi SMS on Treatment Outcomes of Tuberculosis: A Randomized Control Trial Protocol.

https://doi.org/10.1371/journal.pone.0162944.s003

(DOCX)

Acknowledgments

We thank Osman Siddiqi, Zuha Shaikh, Raheel Memon, Faraz Haqqi, Amjad Khan, Kafeel Zaheer, Owais Ali, and Nameer Jamillee for their role implementing this study. We thank our participating clinics, the NTP-Pakistan, and the PTP-Sindh, and especially Dr. Mateen Khan and Saira Khowaja, for their support throughout the study. We thank the International Initiative for Impact Evaluation for financial support. The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Author Contributions

1. Conceptualization: SM RG AJK.

2. Data curation: SM.

3. Formal analysis: SM RG.

4. Funding acquisition: SM RG AJK.

5. Methodology: SM RG.

6. Project administration: SM.

7. Supervision: RG.

8. Writing – original draft: SM.

9. Writing – review & editing: SM RG AJK.

Citation: Mohammed S, Glennerster R, Khan AJ (2016) Impact of a Daily SMS Medication Reminder System on Tuberculosis Treatment Outcomes: A Randomized Controlled Trial. PLoS ONE 11(11): e0162944. https://doi.org/10.1371/journal.pone.0162944

References

1. World Health Organization. Global Tuberculosis Report 2014. 2014. http://apps.who.int/iris/bitstream/10665/137094/1/9789241564809_eng.pdf?ua=1 (accessed May 26, 2015).

2. Volmink J, Garner P. Directly observed therapy for treating tuberculosis. Cochrane Database Syst Rev 2007; 4: CD003343.

3. Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The Effectiveness of Mobile-Health Technology-Based Health Behaviour Change or Disease Management Interventions for Health Care Consumers: A Systematic Review. PLoS Med 2013; 10:1: e1001363. pmid:23458994

4. Anglada-Martinez H, Riu-Viladoms G, Martin-Conde M, Rovira-Illamola M, Cotaca-Momblona JM, Codina-Jane C. Does mHealth Increase adherence to medication? Results of a systematic review. Int J Clin Pract 2014; 4: 1–24.

5. Vervloet M, Linn AJ, van Weert JCM, de Bakker DH, Bouvy ML, van Dijk L. The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature. J Am Med Inform Assoc 2012; 19(5): 696–704. pmid:22534082

6. De Jongh T, Gurol-Urganci I, Codopivic-Jamek V, Car J, Atun R. Mobile phone messaging for facilitating self-management of long-term illnesses. Cochrane Database Syst Rev 2012; 12: CD007459. pmid:23235644

7. Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised control trial. Lancet 2010; 376: 1838–45. pmid:21071074

8. Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011; 25(6): 825–34. pmid:21252632

9. Mbuagbaw L, Thabane L, Ongolo-Zogo P, Lester RT, Mills EJ, Smieja M, et al. The Cameroon Mobile SMS (CAMPS) Trail: A Randomized Trial of Text Messaging versus Usual Care for Adherence to Antiretroviral Therapy. PLoS One 2012; 7(12): 6–12.

10. Finitsis DJ, Pellowski JA, Johnson BT. Text message intervention designs to promote adherence to antiretroviral therapy (ART): A meta-analysis of randomized controlled trials. PLoS One. 2014; 9(2).

11. Mbuagbaw L, van der Kop ML, Lester RT, Thirumurthy H, Pop-Eleches C, Ye C, et al. Mobile phone text messages for improving adherence to antiretroviral therapy (ART): an individual patient data meta-analysis of randomised trials. BMJ Open 2013; 3(12):e003950. pmid:24345901

12. Hovarth T, Azman H, Kennedy GE, Rutherford GW. Mobile phone text messaging for promoting adherence to antiretroviral therapy in patients with HIV infection. Cochrane Database Syst Rev 2012; 3:CD009756.

13. Shet A, De Costa A, Kumarasamy N, Rodrigues R, Rewari BB, Ashorn P, et al. Effect of mobile phone reminders on treatment outcome in HIV: evidence from a randomised controlled trial in India. BMJ 2014; 347: g5978.

14. Ahmed J, Skarbek S, Codlin A, Khan A, Mohaupt D. mHealth to improve TB care. Interactive Research and Development Karachi. 2012. http://www.stoptb.org/assets/documents/resources/publications/acsm/mHealth%20to%20Improve%20TB%20Care.pdf (accessed May 26, 2015).

15. Nglazi MD, Bekker L-G, Wood R, Hussey GD, Wiysonge CS. Mobile phone text messaging for promoting adherence to anti-tuberculosis treatment: a systematic review. BMC Infect Dis 2013; 13:566. pmid:24295439

16. Barclay E. Text messages could hasten tuberculosis drug compliance. Lancet 2009. 373: 15–6. pmid:19125443

17. Denkinger CM, Genier J, Stratis AK, Akkihal A, Pant-Pai N, Pai M. Mobile health to improve tuberculosis care and control: A call worth making. Int J Tuberc Lung Dis 2013; 17: 719–27. pmid:23541232

18. Liu X, Lewis JJ, Zhang H, Lu W, Zhang S, Zheng G, et al. Effectiveness of Electronic Reminders to Improve Medication Adherence in Tuberculosis Patients: A Cluster-Randomised Trial. PLoS Med 2015; 12(9): e1001876. pmid:26372470

19. Mohammed S, Siddiqi O, Ali O, Habib A, Haqqi F, Kausar M, et al. User engagement with and attitudes towards an interactive SMS reminder system for patients with tuberculosis. J Telemed Telecare 2012; 18(7) 404–8. pmid:23034935

20. Westfall P, Young S. Resampling-Based Multiple Testing. New York: Wiley; 1993.

21. Lester RT. Ask, Don’t Tell—Mobile Phones to Improve HIV Care. N Engl J Med 2013; 369: 1867–8.

22. Mills EJ, Lester R, Thorlund K, Lorenzi M, Muldoon K, Kanters S, et al. Interventions to promote adherence to antiretroviral therapy in Africa: a network meta-analysis. Lancet HIV. Elsevier Ltd; 2014;1(3):e104–11. pmid:26424119

23. Liu Q, Abba K, Alejandria MM, Sinclair D, Balanag VM, Lansang MA. Reminder systems to improve patient adherence to tuberculosis clinic appointments for diganosis and treatment. Cochrane Database Syst Rev 2014; 11:CD006594.