Assessing mobile technology use and mHealth acceptance among HIV-positive men who have sex with men and transgender women in Malaysia

Abstract

[...]MSM and TGW with HIV are amongst the most vulnerable and underserved PWH in Malaysia and often unable to benefit from access to government-subsidized ART. The growth of communication technology in Malaysia, thus holds promise in being able to incorporate mHealth strategies and tools in abetting health management and influencing behavior change. [...]there is precedence in the literature that indicates that lesbian, gay, bisexual transgender and queer (LGBTQ) individuals are early adopters of information and communication technologies [19, 20], and hence might benefit from mHealth-enabled interventions. A random sample in this context was deemed to be logistically impractical as MSM and TGW are hard-to-reach populations due to considerable social stigma surrounding sexual minorities and HIV in Malaysia. [...]a convenience approach was utilized [22] by approaching patients at the university clinic and using a research assistant familiar with the MSM and TGW population in Kuala Lumpur. Social support was measured on a 5-point Likert scale using Sherbourne and Stewart’s 19-item social support instrument [29]. Since the population in the present study was both HIV-positive and a sexual minority, two types of stigma were assessed–HIV stigma and sexual orientation stigma.

About the Authors:

Archana Krishnan

Roles Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation: Department of Communication, University at Albany, State University of New York, Albany, New York, United States of America

ORCID logo https://orcid.org/0000-0001-6021-7736

Damian Weikum

Roles Conceptualization, Funding acquisition, Project administration, Writing – review & editing

Affiliation: Yale School of Public Health, New Haven, Connecticut, United States of America

Claire Cravero

Roles Methodology, Writing – review & editing

Affiliation: Harvard T.H. Chan School of Public Health, Boston, Massachusetts, United States of America

Adeeba Kamarulzaman

Roles Project administration, Supervision, Writing – review & editing

Affiliation: Centre of Excellence for Research in AIDS (CERiA), Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

Frederick L. Altice

Roles Conceptualization, Funding acquisition, Writing – review & editing

Affiliations Yale School of Public Health, New Haven, Connecticut, United States of America, Centre of Excellence for Research in AIDS (CERiA), Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia, AIDS Program, Section of Infectious Diseases, Yale School of Medicine, New Haven, Connecticut, United States of America

Introduction

Mobile health (mHealth) is a field of research that centers around the use of mobile devices to manage healthcare, improve health outcomes, and promote preventive health behaviors [1–3]. In response to international guidelines that have recommended scaling back HIV adherence interventions in terms of cost and personnel [4], mHealth has been offered as a complement to existing strategies [5]. mHealth interventions have already shown efficacious results in improving health outcomes related to multiple chronic conditions [6–8], and they can be particularly beneficial in managing the complexity of care for people with HIV (PWH). Compared to other health conditions, HIV care is especially burdensome in terms of cost and effort. PWH need to engage in affirmative activities such as adhering daily to keep their viral load at an undetectable level, and refrain from risky injection drug use and sexual behaviors to prevent transmission [9, 10]. There are however, limited feasibility and acceptability studies of mHealth in general, and limited mHealth research among highly vulnerable groups of PWH such as people who use drugs (PWID), men who have sex with men (MSM) and sex workers [5, 11]. Moreover, there is insufficient evidence to inform the implementation and scale-up of mHealth in various disparate HIV populations.

In many countries, the HIV epidemic is concentrated in specific groups; this is particularly true in a low-to-middle-income country (LMIC) such as Malaysia where MSM and transgender women (TGW) have HIV prevalence 12–18 times than that of the general population [12]. While HIV prevalence in the general Malaysian population is low at around 0.4%, prevalence among key groups is disproportionately high, e.g., 8.9% among MSM and 5.6% among TGW [12]. Multiple reasons–high levels of risk behavior, low levels of knowledge about transmission risk, and low rates of HIV testing among MSM–have been suggested for the increase in HIV incidence [13, 14]. MSM also face substantial societal stigma and legal barriers in Malaysia, a trend that is mirrored in other LMICs in the region such as Indonesia and Cambodia [15]. Specifically, section 377 of the Malaysian Penal Code criminalizes same-sex intercourse and levies imprisonment of up to 20 years. Homosexuality and cross-dressing are also illegal in Shariah law in Malaysia. Recent studies have found high levels of stigma against HIV-positive MSM even amongst medical students and healthcare providers [16, 17]. As a result, MSM and TGW with HIV are amongst the most vulnerable and underserved PWH in Malaysia and often unable to benefit from access to government-subsidized ART.

Malaysia is a fair representation of LMICs both in the Southeast Asian region and globally, with respect to telecommunication infrastructure and consumer base. The adoption of telecommunications in Malaysia, like in the rest of Southeast Asia, has been rapid. Recent reports indicate that Malaysia has a mobile phone penetration rate of 97.5% and Internet penetration of 71.1% [18]. The growth of communication technology in Malaysia, thus holds promise in being able to incorporate mHealth strategies and tools in abetting health management and influencing behavior change. Moreover, there is precedence in the literature that indicates that lesbian, gay, bisexual transgender and queer (LGBTQ) individuals are early adopters of information and communication technologies [19, 20], and hence might benefit from mHealth-enabled interventions. There is however, no published research on the use of mobile technologies and acceptance of mHealth among vulnerable HIV populations in Asia. An ideal opportunity exists in Malaysia to conduct mHealth feasibility work. In response to criticism that mHealth is often implemented in the face of a lack of acceptability work [5, 21] and in recognizing the importance of examining mHealth among vulnerable populations [11], the purpose of this study is to ascertain at a micro level the following: (1) mobile technology access and use, (2) mHealth intervention acceptability, and (3) social and health constructs including drug/alcohol use, mental health and stigma, among HIV-positive MSM and TGW in Kuala Lumpur, Malaysia. The findings will allow us to assess opportunities to implement future mHealth interventions in Malaysia to improve HIV outcomes.

Methodology

Study sample and setting

A purposive sample of 150 HIV-positive MSM and TGW was recruited over 12 weeks from the Pusat Perubatan Universiti Malaya clinic and through community outreach in Kuala Lumpur, Malaysia. The Pusat Perubatan clinic is an academic health clinic affiliated with the University of Malaya which provides medical care to the general public and to key populations such as PWH and LGBTQ. A random sample in this context was deemed to be logistically impractical as MSM and TGW are hard-to-reach populations due to considerable social stigma surrounding sexual minorities and HIV in Malaysia. Thus, a convenience approach was utilized [22] by approaching patients at the university clinic and using a research assistant familiar with the MSM and TGW population in Kuala Lumpur. Recruitment was limited to individuals who: 1) self-identified as MSM or TGW, 2) were 18 years or older, 3) were HIV-positive, 4) had been receiving HIV care for at least 12 months prior to study enrollment, and 5) had been prescribed ART 6 or more months prior to study enrollment.

Procedure

Recruitment included clinic-based advertisements and flyers or clinical staff approaching patients at the end of their clinical visits. Community outreach recruitment occurred through distribution of advertisements and fliers at adherence support meetings and online advertisement on Facebook. After participants were informed of the study and provided written informed consent, they completed a 60-minute computer-assisted survey interview (CASI) in a private setting and were paid 30 Malaysian Ringgit (∼$8 US) for their time. The survey was developed and hosted on the Qualtrics online survey platform, and was administered by a research assistant on a tablet computer. The study protocol was approved by Yale University Human Investigation Committee and the University of Malaya Medical Ethics Committee (protocol # 1404013880).

Measures

Health variables.

Depression was measured using the 10-item Center for Epidemiologic Studies Depression Scale (CES-D 10) [23] with a cut-off of > 7 to define moderate-to-severe depression. Stress was assessed on a 5-point Likert scale using the 4-item perceived stress scale [24]. Health literacy was assessed using a 3-item screening instrument measured on a 5-point Likert scale [25]. Severity of drug use was measured using the DAST-10 which is a 10-item aggregate score of drug use behaviors. Severity of alcohol use specifically was scored using the World Health Organization’s validated 10-item Alcohol Use Disorders Identification Test (AUDIT) [26]. The aggregate score of 40 was categorized with standard cut-offs to indicate severity of alcohol use, specifically normal drinking (0–7), hazardous drinking (8–15), harmful drinking (16–19) and alcohol dependence (≥ 20). Risky sexual behavior was measured using two items assessing the following behaviors–unprotected anal intercourse during the last sexual encounter and frequency of unprotected sex in the past three months. ART adherence was measured using two items, one which was assessed on a 5-point Likert scale (“How would you rate your ability to take your HIV medications every day over the last month”), and the other assessed on a 0–100% scale (“In the past month, please indicate the percentage of ART medications that you took.”). The latter item was transformed into two binary variables–optimal adherence (≥ 90%) and perfect adherence (100%) based on prior studies [20, 27, 28].

Social variables.

Social support was measured on a 5-point Likert scale using Sherbourne and Stewart’s 19-item social support instrument [29]. Since the population in the present study was both HIV-positive and a sexual minority, two types of stigma were assessed–HIV stigma and sexual orientation stigma. HIV stigma was measured using the Wright et al. (2007)’s 10-item scale [30] which is assessed on a 4-point Likert scale. Sexual orientation stigma was adapted from Wright et al. [30] by rewording the items to reflect sexual orientation instead of HIV.

Technology variables.

Self-created instruments were used to measure mobile phone access measured categorically and as an open-ended item. Frequency of mobile phone use was measured using 14 items on a 5-point Likert scale (e.g., “How often do you engage in sending or receiving text messages on your mobile phone?”). Frequency of internet use was measured using 7 items on a 5-point Likert scale. The items were all specific to internet use for general and HIV-related behaviors (e.g., “Indicate how often you search the web for HIV-related health information.”). The 6-item mHealth acceptance scale was adapted from Krishnan et al. [20] that were measured on a 5-point Likert scale (e.g., “How likely are you to use mHealth in reminding you to take your HIV medication?”). Mean scores were created for the above scales, and reliabilities ranged from good to excellent (see Table 3).

Results

The sample was composed of 76% MSM (n = 114) and 24% TGW (n = 36), and was relatively young (M = 35.1 years, SD = 8.3), well-educated (58.6% college-educated), and employed (82% full-time). 63.3% identified as homosexual, 15.3% as bisexual, and 20.7% as heterosexual. Table 1 shows the number of participants who met the cut-off for moderate to severe depression, unprotected anal sex, risky sexual behavior, and optimal and perfect ART adherence.

[Figure omitted. See PDF.]

Table 1. Medication adherence and HIV risk factors for study participants (N = 150).

https://doi.org/10.1371/journal.pone.0248705.t001

Despite study enrollment being restricted to participants who had been prescribed ART in the past 6 months, only 83.3% were currently taking ART. Although severity of alcohol (6.7%) and drug use (10.7%) were low in this sample, sexual risk was high with a third reporting unprotected sex during the last encounter (27.8%) and in the past six months (30%). Stress, depression and HIV stigma were also relatively high with 42% meeting criteria for moderate to severe depression. On the other hand, the sample had high health literacy (M = 4.29, SD = 0.85) and high levels of social support (M = 3.43, SD = 0.89). Table 2 depicts the mean scores for the social and mental health variables.

[Figure omitted. See PDF.]

Table 2. Mean scores for social and mental health variables (N = 150).

https://doi.org/10.1371/journal.pone.0248705.t002

With respect to mobile technology access, 75.3% had access to smartphones, 28% had access to a tablet, and 20.7% to a standard mobile phone without internet access. Even though only 78% had access to the internet, more than three-fourths accessed it on their mobile devices. Table 3 shows the overall scale and individual item mean scores for frequency of mobile phone use, frequency of internet use, and mHealth acceptance. As can be noted from the table, the most frequent uses of mobile phones were making or receiving phone calls (M = 4.22, SD = 0.76), and sending or receiving text messages (M = 3.99, SD = 0.97). With respect to internet use, the most frequent use was searching for general information (M = 3.44, SD = 1.01) followed by HIV-specific health information searching (M = 2.97, SD = 1.12). Overall, acceptance of mHealth was higher than average (M = 2.61, SD = 1.00), with the highest acceptability being towards using mHealth for receiving information about HIV health (M = 2.95, SD = 1.34) and sexual health (M = 2.93, SD = 1.32), and for receiving reminders to take HIV medications (M = 2.88, SD = 1.65).

[Figure omitted. See PDF.]

Table 3. Means, standard deviations, and reliabilities of mobile technology use, internet use and mHealth acceptance scales.

https://doi.org/10.1371/journal.pone.0248705.t003

To gauge if mental health status of participants acted as a barrier to mobile technology use and mHealth acceptance, multiple regressions were conducted. A significant equation was found predicting mobile phone use from stress and depression at F (2, 136) = 4.70, p < .05, R2 = .07; however, only stress was a significant predictor at t = 3.03, p < .01, β = .25. This significant result shows that stress was not a barrier and in fact was positively associated with frequency of mobile phone use. Significant equations were not found for stress and depression predicting internet use and mHealth acceptance at F (2, 114) = 1.74, p = .18 and F (2, 147) = .12, p = .89 respectively. This indicates that mental health was not a barrier to either internet use or mHealth acceptance.

Suboptimal ART adherence is associated with poor HIV treatment outcomes [31, 32] and is thus a prime outcome for behavioral change interventions. Thus, it is important to assess if acceptability of mHealth varies among those who are optimally adherent or not, and those who are perfectly adherent or not. A series of independent samples t-tests (see Table 4) was conducted to look for significant differences in mHealth acceptance among those who had optimal adherence and perfect adherence. A statistically significant difference was found between those who reported being optimally adherent and those who did not. i.e., those who were not optimally adherent reported higher acceptability of mHealth. No significant difference was found between groups with and without perfect adherence.

[Figure omitted. See PDF.]

Table 4. Differences in mHealth acceptance among optimal and perfect adherence groups.

https://doi.org/10.1371/journal.pone.0248705.t004

Discussion

Our findings provide a micro-level perspective of both the public health and technology make-up of a sample of HIV-positive MSM and TGW living in Malaysia. The sample had high levels of depression and risky sexual behavior which unfortunately is typical in a socially- and medically-underserved HIV population. First-line HIV medications are fully government-subsidized for all Malaysian nationals. However, despite being eligible to receive HIV medication and study enrollment being restricted to participants who had been prescribed ART in the past 6 months, almost a fifth of the sample was not on an ART regimen. This is an indirect measure of healthcare stigma against HIV-positive MSM and TGW, one that is corroborated by high perceived stigma scores in this study and one that has been highlighted by previous research. Earnshaw et al. [16] and Ferro et al. [17] showed high levels of discrimination against PWH in Malaysia by healthcare providers and the detrimental effect it had on treatment intention, especially if PWH were a sexual minority or had co-occurring issues such as alcohol and drug use disorders. Suboptimal levels of ART access among vulnerable populations such as MSM and TGW have been documented elsewhere [27, 28, 33, 34].

Among the participants who were on ART, one-fourth were not optimally adherent to their medication regimen. More troubling, a third reported high-risk sexual behaviors. This reveals the extreme vulnerability of this group of PWH along with a treatment gap that needs to be addressed. Suboptimal ART adherence is associated with incomplete viral suppression which increases HIV transmission risk by those who continue to engage in high-risk sexual behaviors. Achieving optimal ART adherence, as part of the treatment and prevention approach, is thus crucial to reduce HIV transmission and improve survival [35, 36]. This is precisely why most interventions for PWH tend to focus on improving ART adherence. In fact, a meta-analysis found that those who participated in adherence-enhancing interventions were significantly more likely to achieve 95% adherence and viral load suppression [37].

As international guidelines suggest scaling back HIV interventions in terms of cost and personnel [4, 11], mHealth is poised to fulfill this need. This study reveals an opportunity to implement mHealth–a majority of the sample had access to smartphones. In fact, mobile phone ownership rates were higher than that of the general population in Malaysia [18]. Similarly, a majority were accepting of mHealth to assess their drug use and improve their medication adherence among others. More importantly, not only was mental health status not a barrier to using mobile technology and the internet, but mHealth acceptance was higher among those with suboptimal adherence. mHealth can be particularly advantageous in resource-limited settings [7, 38], thus making its use particularly appealing in an emerging economy such as Malaysia. Moreover, the sample had high health literacy and high levels of social support, both of which can add valuable support to adherence interventions.

This study has shown strong preliminary evidence of both a gap and an opportunity that exists in Malaysia, to implement mHealth among underserved PWH. There has not any previous examination in the region that lays out the feasibility of implementing an mHealth intervention among underserved PWH. The findings in this study echo previous findings that showed high acceptability of mHealth among PWH, albeit in other geographical regions [20, 39]. There is strong evidence for an mHealth-based intervention among HIV-positive MSM and TGW who are not optimally adherent to ART. The focus of future mHealth interventions must be both on decreasing risky sexual behavior and improving ART adherence, both of which can be monitored in real-time or near real-time and influenced by disseminating tailored health communication messages. Furthermore, since low rates of HIV testing are partially to blame for rising rates among MSM, ubiquitous mobile phone use can be leveraged to disseminate information about HIV testing.

Key HIV populations like MSM and TGW are not only underserved with respect to medical access and care, but efforts to keep them engaged with medication regimens are fraught with stigma and discrimination sometimes by the very people treating them. mHealth interventions can to a certain extent, provide support and protection against discriminatory behavior. Moreover, high levels of technology use and mHealth acceptance among HIV-positive MSM and TGW in Malaysia provide strong evidence for developing and implementing a technology-based adherence intervention. This is likely to address the urgent need for innovative yet cost-effective strategies to address treatment gaps for underserved HIV populations.

Acknowledgments

The authors would like to gratefully acknowledge the research staff at the University of Malaya for assisting with the data collection for this study.

Citation: Krishnan A, Weikum D, Cravero C, Kamarulzaman A, Altice FL (2021) Assessing mobile technology use and mHealth acceptance among HIV-positive men who have sex with men and transgender women in Malaysia. PLoS ONE 16(3): e0248705. https://doi.org/10.1371/journal.pone.0248705

References

1. Lim MSC, Hocking JS, Hellard ME, Aitken CK. SMS STI: A review of the uses of mobile phone text messaging in sexual health. International Journal of STD & AIDS. 2008;19(5):287–90. pmid:18482956

2. Fjeldsoe BS, Marshall AL, Miller YD. Behavior change interventions delivered by mobile telephone short-message service. American Journal of Preventive Medicine. 2009;36(2):165–73. pmid:19135907

3. Katz DL, Nordwall B. Novel interactive cell-phone technology for health enhancement. Journal of diabetes science and technology. 2008;2(1):147–53. pmid:19885191

4. Thompson MA, Mugavero MJ, Amico KR, Cargill VA, Chang LW, Gross R, et al. Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: Evidence-based recommendations from an International Association of Physicians in AIDS Care panel. Ann Intern Med. 2012;156(11):817–33, W-284, W-5, W-6, W-7, W-8, W-9, W-90, W-91, W-92, W-93, W-94. pmid:22393036

5. Krishnan A, Cravero C. A multipronged evidence-based approach to implement mHealth for underserved HIV-infected populations. Mobile Media & Communication. 2017;5(2):194–211.

6. Hoffman JA, Cunningham JR, Suleh AJ, Sundsmo A, Dekker D, Vago F, et al. Mobile direct observation treatment for Tuberculosis patients: A technical feasibility pilot using mobile phones in Nairobi, Kenya. American Journal of Preventive Medicine. 2010;39(1):78–80. pmid:20537846

7. Leach-Lemens C. Using mobile phones in HIV care and prevention. HIV & AIDS Treatment in Practice. 2009;137:7.

8. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-management of adolescent type 1 diabetes: A pilot study. J Med Internet Res. 2012;14(3):e70. pmid:22564332

9. Ammassari A, Trotta MP, Murri R, Castelli F, Narciso P, Noto P, et al. Correlates and predictors of adherence to highly active antiretroviral therapy: Overview of published literature. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2002;31:S123–S7. pmid:12562034

10. Arnsten JH, Demas PA, Grant RW, Gourevitch MN, Farzadegan H, Howard AA, et al. Impact of active drug use on antiretroviral therapy adherence and viral suppression in HIV-infected drug users. J Gen Intern Med. 2002;17(5):377–81. pmid:12047736

11. Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV teatment and prevention: A systematic review of the literature. Open AIDS J. 2013;7:17–41. pmid:24133558

12. Malaysia Ministry of Health. Global AIDS Response Progress Report 2016. HIV/STI Section,Malaysia Ministry of Health, 2016.

13. Guadamuz TE, Cheung DH, Wei C, Koe S, Lim SH. Young, Online and in the dark: Scaling up HIV testing among MSM in ASEAN. PLOS ONE. 2015;10(5):e0126658. pmid:25973907

14. Kanter J, Koh C, Razali K, Tai R, Izenberg J, Rajan L, et al. Risk behaviour and HIV prevalence among men who have sex with men in a multiethnic society: A venue-based study in Kuala Lumpur, Malaysia. Int J STD AIDS. 2011;22(1):30–7. pmid:21364064

15. NQAPIA. Fact sheet: The state of LGBTQ rights in Asia and the Pacific 2018. Available from: https://www.nqapia.org/wpp/state-of-lgbtq-rights-in-asia-and-the-pacific/.

16. Earnshaw VA, Jin H, Wickersham J, Kamarulzaman A, John J, Altice FL. Exploring intentions to discriminate against patients living with HIV/AIDS among future healthcare providers in Malaysia. Tropical Medicine & International health: TM & IH. 2014;19(6):672–9. pmid:24666546

17. Ferro EG, Culbert GJ, Wickersham JA, Marcus R, Steffen AD, Pauls HA, et al. Physician decisions to defer antiretroviral therapy in key populations: Implications for reducing human immunodeficiency virus incidence and mortality in Malaysia. Open Forum Infect Dis. 2017;4(1):ofw219. pmid:28480230

18. Malaysia Department of Statistics. ICT Use and Access By Individuals and Households Survey Report, Malaysia, 2015. Statistics MDo; 2015.

19. Bolding G, Davis M, Sherr L, Hart G, Elford J. Use of gay Internet sites and views about online health promotion among men who have sex with men. AIDS Care. 2004;16(8):993–1001. pmid:15511731

20. Krishnan A, Ferro EG, Weikum D, Vagenas P, Lama JR, Sanchez J, et al. Communication technology use and mHealth acceptance among HIV-infected men who have sex with men in Peru: Implications for HIV prevention and treatment. AIDS Care. 2015;27(3):273–82. pmid:25285464

21. Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: Where is the evidence? PLoS Med. 2013;10(2):e1001382. pmid:23424286

22. Magnani R, Sabin K, Saidel T, Heckathorn D. Review of sampling hard-to-reach and hidden populations for HIV surveillance. AIDS. 2005;19:S67–S72. pmid:15930843

23. Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1(3):385–401.

24. Cohen S, Williamson G. Perceived stress in a probability sample of the US. In: Spacapam S, Oskamp S, editors. The social psychology of health: Claremont Symposium on Applied Social Psychology: Newbury Park, CA: Sage; 1988. p. 31–67.

25. Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Family medicine. 2004;36(8):588–94. pmid:15343421

26. Babor TF, Higgins-Biddle JC, Saunders JB, Monteiro MG. AUDIT: The Alcohol Use Disorders Identification Test. Geneva, Switzerland: World Health Organization (W.H.O.), 2001.

27. Rich KM, Valencia Huamani J, Kiani SN, Cabello R, Elish P, Florez Arce J, et al. Correlates of viral suppression among HIV-infected men who have sex with men and transgender women in Lima, Peru. AIDS Care. 2018;30(11):1341–50. pmid:29843518

28. Rich KM, Wickersham JA, Valencia Huamani J, Kiani SN, Cabello R, Elish P, et al. Factors Associated with HIV Viral Suppression Among Transgender Women in Lima, Peru. LGBT Health. 2018;5(8):477–83. pmid:30874476

29. Sherbourne CD, Stewart AL. The MOS Social Support Survey. Social Science Medicine. 1991;32(6):705–14. pmid:2035047

30. Wright K, Naar-King S, Lam P, Templin T, Frey M. Stigma scale revised: Reliability and validity of a brief measure of stigma for HIV+ youth. Journal of Adolescent Health. 2007;40(1):96–8. pmid:17185215

31. Altice FL, Bruce RD, Lucas GM, Lum PJ, Korthuis PT, Flanigan TP, et al. HIV treatment outcomes among HIV-infected, opioid-dependent patients receiving buprenorphine/naloxone treatment within HIV clinical care settings: Results from a multisite study. J Acquir Immune Defic Syndr. 2011;56 Suppl 1:S22–32. pmid:21317590

32. Maru DS, Bruce RD, Walton M, Mezger JA, Springer SA, Shield D, et al. Initiation, adherence, and retention in a randomized controlled trial of directly administered antiretroviral therapy. AIDS Behav. 2008;12(2):284–93. pmid:18085432

33. Wood E, Montaner JS, Bangsberg DR, Tyndall MW, Strathdee SA, O’Shaughnessy MV, et al. Expanding access to HIV antiretroviral therapy among marginalized populations in the developed world. AIDS (London, England). 2003;17(17):2419–27. pmid:14600512

34. Wolfe D, Carrieri MP, Shepard D. Treatment and care for injecting drug users with HIV infection: a review of barriers and ways forward. Lancet. 2010;376(9738):355–66. pmid:20650513

35. Palella FJ Jr., Delaney KM, Moorman AC, Loveless MO, Fuhrer J, Satten GA, et al. Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. HIV Outpatient Study Investigators. N Engl J Med. 1998;338(13):853–60. pmid:9516219

36. Paterson DL, Swindells S, Mohr J, Brester M, Vergis EN, Squier C, et al. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Annals of Internal Medicine. 2000;133(1):21–30. pmid:10877736

37. Simoni JM, Pearson CR, Pantalone DW, Marks G, Crepaz N. Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV-1 RNA viral load: a meta-analytic review of randomized controlled trials. Journal of Acquired Immune Deficiency Syndromes (1999). 2006;43(0 1):S23. pmid:17133201

38. Karanja S, Mbuagbaw L, Ritvo P, Law J, Kyobutungi C, Reid G, et al. A workshop report on HIV mHealth synergy and strategy meeting to review emerging evidence-based mHealth interventions and develop a framework for scale-up of these interventions. The Pan African Medical Journal. 2011;10:37. pmid:22187619

39. Miller CW, Himelhoch S. Acceptability of mobile phone technology for medication adherence interventions among HIV-positive patients at an urban clinic. AIDS Research and Treatment. 2013;2013:670525. pmid:23997948