Self-reported adherence and pharmacy refill adherence are both predictive for an undetectable viral load among HIV-infected migrants receiving cART

Abstract

HIV-infected migrants were shown to have poorer treatment outcomes than Dutch HIV-infected patients, often due to worse treatment adherence. Self-reported adherence would be an easy way to monitor adherence, but its validity relative to pharmacy refill adherence has not been extensively evaluated in migrants. All HIV-infected migrants older than 18 years and in care at the two Rotterdam HIV-treatment centers were eligible. Refill data with leftover medication (PRL) (residual pill count) were obtained from their pharmacies up to 15 months prior to inclusion. Self-reported adherence to combination Antiretroviral Therapy was assessed by four questions about adherence at inclusion. Additionally, risk factors for pharmacy refill non-adherence were examined. In total, 299 HIV-infected migrants were included. Viral load (VL) was detectable in 11% of the patients. Specificity of PRL was 53% for patients with an adherence of 100% and decreased with lower cut-off values. Sensitivity and negative predictive value (NPV) were 68% and 15% and increased with lower cut-off values. Positive predictive value (PPV) was around 93% for all cut-off values. Using the self-reported questions, 139 patients (47%) reported to be adherent. Sensitivity was 49% and specificity was 72%. PPV and NPV were 95% and 13%. No risk factors for pharmacy refill non-adherence were found in multivariable analyses. Both PRL and self-reported adherence, can predict undetectable VL in HIV-infected migrants. PPV and NPV are similar for both methods. This study shows that using four self-reported items is sufficient to predict adherence which is crucial for optimal clinical outcome in HIV-infected migrants.

About the Authors:

Sabrina K. Been

Contributed equally to this work with: Sabrina K. Been, Elif Yildiz, Annelies Verbon

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected] (SB); [email protected] (AV)

Affiliation: Department of Internal Medicine, division of Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands

ORCID http://orcid.org/0000-0002-5755-3306

Elif Yildiz

Contributed equally to this work with: Sabrina K. Been, Elif Yildiz, Annelies Verbon

Roles Conceptualization, Data curation, Formal analysis, Investigation, Visualization, Writing – original draft, Writing – review & editing

Affiliations Department of Internal Medicine, division of Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands, Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands

Pythia T. Nieuwkerk

Roles Conceptualization, Formal analysis, Methodology, Project administration, Resources, Visualization, Writing – review & editing

¶‡ These authors also contributed equally to this work

Affiliation: Department of Medical Psychology, Academic Medical Center, Amsterdam, The Netherlands

Katalin Pogány

Roles Resources, Writing – review & editing

¶‡ These authors also contributed equally to this work

Affiliation: Department of Internal Medicine, Maasstad Hospital, Rotterdam, The Netherlands

David A. M. C. van de Vijver

Roles Formal analysis, Methodology, Visualization, Writing – review & editing

¶‡ These authors also contributed equally to this work

Affiliation: Viroscience department, Erasmus University Medical Center, Rotterdam, The Netherlands

Annelies Verbon

Contributed equally to this work with: Sabrina K. Been, Elif Yildiz, Annelies Verbon

Roles Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected] (SB); [email protected] (AV)

Affiliations Department of Internal Medicine, division of Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands, Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands

Introduction

Treatment adherence is one of the strongest predictors of virological failure, development of drug resistance, disease progression and death [1]. Poor adherence to combination antiretroviral therapy (cART) is common in both developing and developed nations. It was found in around 20% of HIV-infected patients in Africa and in around 14% in the United Stated of America [2–4]. It is not clear what the influence of migration from developing countries to developed countries is on adherence and outcome. In Europe, migrant people living with HIV (MPLWH) have worse treatment outcomes than non-migrant patients living with HIV [5–9]. Although lack of adherence could not be demonstrated as a reason for virological failure in one study [6], factors associated with poor adherence such as younger age, high alcohol consumption, presence of psychiatric co-morbidity, stigma, lack of self-efficacy or social support are frequently found in migrants [6, 10, 11]. With the strongly increasing migration from Africa and the Middle East to Europe it is pivotal to determine adherence levels with the goal to intervene when adherence is poor in order to improve treatment outcome in MPLWH.

Treatment adherence of HIV-infected patients can be measured in several ways. Pharmacy refill data have been shown to be a useful tool in both developing and developed nations [2, 12–14]. Calculation of pharmacy refill adherence is most practical in closed pharmacy systems where information about dispensed medication can be retrieved from a single registry. In the absence of a closed pharmacy system, calculation of pharmacy refill adherence is more laborious because information retrieved from different pharmacies need to be combined. Although pharmacy refill data are valuable on a population level, it seems to be less suitable to measure adherence on an individual patient level. Self-reported adherence questions, administered either via questionnaires or via interviews, have previously been shown to be a practical adherence assessment method [6, 14]. However, it is well established that self-reports may overestimate adherence levels compared with more objective measures, which is attributed to recall bias and social desirability bias [15]. The validity of self-reported adherence questions administered among MPLWH in Europe might be influenced by cultural differences. This could facilitate social desirability bias.

Since retention in care and treatment adherence are crucial for a good outcome in HIV-infected patients [12, 16], and MPLWH were previously shown to have worse HIV treatment outcomes and adherence levels, valid and practical adherence measures are needed to identify persons in need of additional adherence support. We therefore assessed two methods, pharmacy refill adherence and self-reported adherence, in order to determine their comparative predictive value for undetectable viral load in a large group of MPLWH from different geographical origins. In addition, we examined risk factors for non-adherence based on pharmacy refill data calculation.

Methods

Study population

Patients included in the ROtterdam ADherence (ROAD) project, a study aiming to increase adherence in first and second generation adult immigrant people living with HIV (The Nederlands National Trial Register Number (NTR)—NTR4941) were recruited at outpatient clinics in the two Rotterdam HIV treatment centers (Erasmus University Medical Center and Maasstad Hospital) between November 2012 and July 2013 [11]. In these HIV treatment centers, 52% of the PLWH in care in 2014 were first generation immigrants (data not shown). Participants had to be sufficiently fluent in at least one of the following languages: Dutch, English, French, Spanish, or Portuguese.

This study focuses on a sub population of the 352 patients included in the ROAD project. Patients using cART for >6 months prior to inclusion signed informed consent allowing the investigators to consult their pharmacies about medication refills and to check the patients’ medical files for information about the prescribed start and stop of cART regimens, and viral load results. Additional relevant clinical and socio-demographic characteristics, and psychosocial variables were collected via an interviewer administered questionnaire [11]. An undetectable viral load (VL) was defined as HIV-1 RNA or HIV-2 RNA <50 copies/ml.

The study was approved by the Institutional Review Boards of the Erasmus University Medical Center and the Maasstad Hospital, Rotterdam.

Study design and data collection

Collecting pharmacy refill data.

In the Netherlands, there is no closed pharmacy system. Therefore, names of the pharmacies from which antiretroviral medication was collected were given by the participants when they attended the outpatient clinic for a routine visit. In the Netherlands, patients who are stable on cART usually collect their antiretroviral drugs at the pharmacy every 90 days. Pharmacy refill data was collected at one time point. We sent a fax to each patient’s pharmacy to ask for information regarding the type of antiretroviral medication, pills supplied, prescribed daily dose, date of supply up to 15 months prior to inclusion. A copy of the signed informed consent was provided along with the request. Reminders per email and fax were sent and if necessary, pharmacies were contacted by telephone.

Pharmacy refill adherence calculation.

For our analyses, we assumed that medication was taken as prescribed until the supply was finished. Calculation of adherence with pharmacy refill data was done taking leftover medication (residual pill count) into account, a method that substantially limits misclassification of adherence [13]. In short, a period of 3 months prior to inclusion (90 days) was used as the episode to monitor adherence. Pharmacy refill adherence was calculated by determining the average supply percentage of all cART during this 90 days episode. The 12 months (360 days) prior to this episode were used to calculate the leftover medication. Antiretroviral regimen changes were taken into account. For the calculation of leftover medication, we assumed that patients had leftover medication if medication was refilled while the number of pills dispensed at a former refill divided by the prescribed daily doses and multiplied by the days since the previous refill was higher than zero, except when the patient changed his or her regimen. The residual number of pills was rolled over to the next refill period leading to the following formula for pharmacy refill adherence with leftover medication:

If the supply of pills for a refill period extended beyond the 90-day episode or the date of a switch, we truncated adherence at 100%. Additionally, we investigated the number of patients who would be misclassified as ‘non- adherent’ when leftover medication was not taken into account, using the following formula to determine pharmacy refill adherence:

Self-reported adherence.

Self-reported adherence was measured by four items via an interviewer administered questionnaire at the day of inclusion. The questionnaire was available in 4 languages: Dutch, English, French, and Spanish. The first two items measured recollection of adherence: Q1: ‘Thinking about the past four weeks, how would you rate your ability to take all your medications as your doctor prescribed them?’ (6-point Likert scale ranging from 1 = ‘very poor’ to 6 = ‘excellent’) and Q2: ‘Thinking about the past four weeks, how often did you take all your HIV antiretroviral medications as your doctor prescribed them?’ (5-point Likert scale ranging from 1 = ‘none of the time’ to ‘all of the time) [17]. Item three measured the adherence over the past week: Q3: ‘How many days in the past week did you take all anti-HIV medicines that were prescribed?’ (‘5-point Likert scale ranging from 1 = ‘not on day’ to 5 = ‘all 7 days’) [15, 18, 19]. The last item measured the most recent missed dose: Q4: ‘When was the last time you missed any of your anti- HIV medications?’ (6-point Likert scale ranging from 1 = ‘ within the past week’ to ‘never missed’ [20]. For the calculation of adherence, a patient was classified as ‘adherent’ if they had the following answers:

- Q1: ‘very good’ or ‘excellent’

- Q2: ‘all of the time’

- Q3: ‘all 7 days’

- Q4: ‘more than three months ago’, ‘never missed’ or ‘I’m not sure’

Data and statistical analysis

Chi2 and Fisher-Exact tests were used to compare categorical data between groups and T-tests and Mann-Whitney tests for continuous data. For each adherence calculation we calculated specificity (i.e., percentage of patients with detectable VL which is identified as non-adherent), sensitivity (i.e., percentage of patients with undetectable VL which is identified as adherent), negative predictive value (NPV) (i.e., percentage of non-adherent patients that has a detectable VL), and positive predictive value (PPV) (i.e., percentage of adherent patients that has an undetectable VL). Specificity, sensitivity, NPV and PPV were determined for PRL and PR from ≤70% to ≤100%. With logistic regression analyses, independent risk factors for non-adherence calculated with PRL were determined. Risk factors for self-reported non-adherence within this patient population were previously reported [11]. Cut off values for pharmacy refill data adherence were set at 95% and 80%. The 95% cut off value is the gold-standard threshold for older cART regimens, the 80% cut off value is based on a study showing that this threshold appears sufficient to maintain virologic suppression with newer cART regimens [16]. Variables included in the analyses were: VL, gender, region of origin, sexual orientation, family situation, marital status, educational attainment, employment status, alcohol and substance use, social support, HIV-related stigma, treatment adherence self-efficacy, and physical and mental quality of life [11]. For these analyses, scores from the psychosocial variables (social support, internalized HIV-related stigma, treatment adherence self-efficacy, and physical and mental quality of life) were dichotomized based on their median values. In multivariable analysis, we submitted all variables demonstrating a P<0.15 in the univariable analyses.

Results

Patient population

In total, 301 MPLWH used cART for >6 months at inclusion. Of these patients two withdrew their informed consent for contacting their pharmacy, thus 299 patients were included. About 57% were male (n = 112), mean age was 43.2 (SD 10.2) and mean of years on cART was 4.5 (SD 3.3) (Table 1).The majority of the population originated from Sub Saharan Africa region (42.1%). Most of the patients were heterosexual (65.2%), lived with family (39.5%) and were unmarried (58.9%). Median CD4+ cell count was 550*10^6/L (IQR 420.0–710.0) and 11% had a detectable VL.

[Figure omitted. See PDF.]

Table 1. Patient characteristics.

https://doi.org/10.1371/journal.pone.0186912.t001

Of the 299 patients, 245 provided one pharmacy address, 39 provided two pharmacy addresses, and four provided three pharmacy addresses. Patient pharmacies were unknown for 11 of the participants, because they could not remember the name of their pharmacy or were not known at the pharmacy that they had indicated. Pharmacies did not respond to repeated requests for data for 24 patients. For 31 patients data could not be used, for instance because medication was delivered at home. Of the remaining 233 patients, data about leftover medication was not available for 37, leaving for a total of 196 patients with complete data.

These 196 patients (complete data, CD) were significantly older (44.9 years, SD 9.5) than the 103 of whom the pharmacy refill data could not be fully retrieved (ID) (39.9 years, SD 10.9) (Table 1). In addition, the mean number of years on cART was significantly higher in the CD group (5 years) compared to the ID group (3.5 years). Regimens containing nucleoside reverse transcriptase inhibitors (NRTI’s) plus protease inhibitors (PI’s) were used less frequently in the CD group (25.5%) compared to the ID group (35.9%). Furthermore, patients in the CD group originated less often from the Caribbean (19.4% versus 23.3%). The two groups did not differ significantly in self-reported adherence percentages.

Adherence using pharmacy refill data and self-reporting

Table 2 shows the number of patients who were classified as adherent with cut off values of ≤70% to ≤100%, for calculations including leftover medication (PRL), and for calculations excluding leftover medication (PR). Using a cut off value of ≤100% in the PRL group, 65.8% was classified as adherent compared to 44.6% in the PR group. Lowering the cut off value increased the percentage classified as adherent. Specificity was 52.6% for patients with PRL ≤100% and decreased with lower adherence cut off values. Higher cut off values had a lower sensitivity. PPV was around 93% for all cut off values, while NPV increased when lower cut off values were used.

[Figure omitted. See PDF.]

Table 2. Comparison of adherence measures.

https://doi.org/10.1371/journal.pone.0186912.t002

Using the self-reported adherence questions among the patients who had a self-reported adherence outcome (n = 296), 139/296 (47.0%) reported to be adherent (Table 2). When only participants with both a self-reported adherence outcome and a PRL outcome were included in the analyses, 92/195 (46.9%) reported to be adherent. Compared to the self-reported adherence measure, specificity was lower and sensitivity of the pharmacy refill measure was higher with decreasing PRL cut off values. There was no significant difference in PPV, and NPV between PRL using the higher cut off values and self-reported adherence.

Factors associated with non-adherence

Factors associated with <95% pharmacy refill adherence in the univariable analyses were: having a detectable VL, having a ‘other’ type of employment status (vs paid employment), having a low physical quality of life, and having a low mental quality of live (data not shown). Variables demonstrating a P<0.15 were: experiencing low social support, and high internalized stigma. Factors associated with <80% pharmacy refill adherence were having a detectable VL, being on sick leave, having a ‘other’ type of employment status (vs paid employment), drug use in the previous 30 days, and having a low physical quality of life (data not shown). Being on a regimen containing a NRTI plus PI (vs a regimen containing NRTI plus NNRTI) demonstrated a P<0.15.

In the multivariable analyses (Table 3), having a ‘other’ type of employment status (vs paid employment), a low physical quality of life, and a low mental quality of life were marginally significantly associated with <95% pharmacy refill adherence (P<0.10). Having a detectable VL persisted to be significantly associated to a <80% pharmacy refill adherence, as was being on sick leave (vs. paid employment. Having an ‘other’ type of employment status (vs paid employment) was marginally significantly associated with <80% pharmacy refill adherence.

[Figure omitted. See PDF.]

Table 3. Factors related to PRL non-adherencea.

https://doi.org/10.1371/journal.pone.0186912.t003

Discussion

The HIV-infected population in many western European countries increasingly consists of migrants from countries where HIV-infection is highly prevalent. Practical methods to assess adherence in this population are needed as migrants are at risk of having a lower adherence and suboptimal virological response. Our study demonstrates that pharmacy refill adherence and four self-report adherence questions, can predict undetectable VL. Self-reported adherence questions have higher specificity and lower sensitivity compared to PRL. PPV and NPV are similar for both methods. In addition, having an ‘other’ type of employment status is associated with <95% and having a detectable VL with 80% pharmacy refill adherence.

HIV-RNA was detectable in 11% of HIV-infected migrants on cART. This percentage of viremic patients is similar to that found in other studies measuring the association between adherence and virological failure [21, 22], but is lower than previously found in Dutch MPLWH [6]. It is not clearly defined what the gold-standard for adherence studies in HIV-infected patients is. Some studies use pharmacy refill data as golden standard to asses self-reported adherence [23], but most studies use virologic failure [6, 16, 21, 24]. However, incomplete adherence is associated with residual HIV-1 replication, even in the absence of plasma viremia [25]. Because the clinical goal of treatment with cART is to achieve or maintain undetectable plasma viral load, we compared self-reported- and pharmacy refill adherence in their ability to predict viral suppression.

An association between having an ‘other’ type of employment status with <95% pharmacy refill adherence was found. A total of 38 patients were in this group, of whom 15 were identified to be <95% adherent. Most reported to be a housewife/-man (n = 7) or student (n = 4). Being on sick leave was associated with <80% pharmacy refill adherence. A total of 25 were in this group, of whom 6 were identified to be <80% adherent. A possible reason for these associations might be a less structured lifestyle, although this remains speculative.

Of the patients showing 100% PRL adherence, 21.2% was misclassified as non-adherent when leftover medication was not taken into account. This supports the results from another study where a high percentage was misclassified as non-adherent [13]. Therefore we advise to include leftover medication from previous refills in future studies calculating adherence to cART based on pharmacy refill data in countries that do not have a closed pharmacy system.

Defining adherence using pharmacy refill data is challenging. With older cART regimens, a threshold of 95% has been used in clinical practice. Recently it has been shown that for three regimens (emtricitabine-tenofovir plus efavirenz or raltegravir or boosted protease inhibitor (darunavir or atazanavir)) 80% adherence resulted in no more than 3.5% virological failure, whereas adherence of >90% resulted in 1.1% virological failure [16]. The majority of our patients (n = 284, 95%) used a nucleoside reverse transcriptase inhibitor (NRTI) plus a non-nucleoside reverse transcriptase inhibitor (NNRTI) or protease inhibitor (PI) (Table 1) and our results support the suggestion that a pharmacy refill adherence of 80–90% might be sufficient to maintain virologic suppression with the newer cART regimens.

Self-reported adherence is a cheap and easy way to measure adherence. Self-reported adherence using elaborate questionnaires with over 19 items has been shown to predict virological suppression in HIV-patients in high resource settings [6, 16]. In low resource settings, self-reported adherence questions similar to those used in the present study were reported to predict detectable viral load [14]. However, a questionnaire with a visual analog score scale for missed doses and 2 questions regarding missed doses [26] was found to be inferior to pharmacy refill data in predicting virologic failure [22]. Little is known about the validity of self-reported adherence among migrant populations. Self-reported adherence may be biased due to cultural differences or language barriers. Using different questionnaires for adherence, migrants using cART showed a tendency to provide more socially desirable answers [6]. Social desirability bias could even be more pronounced if adherence questions are administered during face-to-face interviews due to literacy problems. However, using a 4-item questionnaire, we show that self-reported adherence predicts virologic suppression in MPLWH treated in a high resource setting. In MPLWH the 4-item questionnaire seems an easy way to identify non-adherent patients and to further determine adherence and barriers to adherence.

The strength of our study is that it is a real life study involving MPLWH using cART during a wide range of time. Our results may be extrapolated to similar clinical settings. Our study should be viewed in the context of some limitations. We measured adherence, but not the barriers to adherence. In an analysis of 11 AIDS Clinical Trials Group studies it was shown that patients reporting a higher number of adherence barriers had lower odds of attaining virologic suppression [27]. The most commonly mentioned adherence barriers were being away from home or forgetting to take the pills. Adherence barriers may differ between different HIV-infected populations. In MPLWH, internalized stigma was associated with non-adherence [10]. Therefore, after identification of MPLWH that are non-adherent, barriers should be assessed to start successful interventions. Another limitation is that we did not evaluate whether non-adherence is associated with the use of certain cART regimens. However, most patients in the Netherlands, including migrants, use the newer cART regimens and since we wanted to study our adherence measures in real life, we choose not to break the results down to the different cART regimens.

In conclusion, in MPLWH both methods, PRL and self-reported adherence questions, can predict undetectable VL. Using self-reported adherence questions results in a lower sensitivity and a higher specificity compared to PRL. PPV and NPV are similar for both methods. Collecting PRL of MPLWH in a country with no closed pharmacy system is laborious and not possible for all patients. This study shows that using four items in self-reported adherence is sufficient to predict adherence which is crucial for optimal clinical outcome in HIV-infected migrants.

Acknowledgments

The authors thank the participants for their time and willingness to contribute to this study. In addition, we thank the physicians and nurses at Erasmus University Medical Center and Maasstad Hospital, and the members of the interview team, for their efforts during the inclusion period. Lastly, we thank our colleagues from the ATHENA observational HIV cohort for contributing to the data collection.

Citation: Been SK, Yildiz E, Nieuwkerk PT, Pogány K, van de Vijver DAMC, Verbon A (2017) Self-reported adherence and pharmacy refill adherence are both predictive for an undetectable viral load among HIV-infected migrants receiving cART. PLoS ONE12(11): e0186912. https://doi.org/10.1371/journal.pone.0186912

References

1. Lima VD, Harrigan R, Bangsberg DR, Hogg RS, Gross R, Yip B, et al. The combined effect of modern highly active antiretroviral therapy regimens and adherence on mortality over time. J Acquir Immune Defic Syndr. 2009;50(5):529–36. pmid:19223785

2. Ochieng W, Kitawi RC, Nzomo TJ, Mwatelah RS, Kimulwo MJ, Ochieng DJ, et al. Implementation and Operational Research: Correlates of Adherence and Treatment Failure Among Kenyan Patients on Long-term Highly Active Antiretroviral Therapy. J Acquir Immune Defic Syndr. 2015;69(2):e49–56. pmid:26009836

3. Beer L, Skarbinski J. Adherence to antiretroviral therapy among HIV-infected adults in the United States. AIDS Educ Prev. 2014;26(6):521–37. pmid:25490733

4. Safren SA, Mayer KH, Ou SS, McCauley M, Grinsztejn B, Hosseinipour MC, et al. Adherence to Early Antiretroviral Therapy: Results From HPTN 052, a Phase III, Multinational Randomized Trial of ART to Prevent HIV-1 Sexual Transmission in Serodiscordant Couples. J Acquir Immune Defic Syndr. 2015;69(2):234–40. pmid:26009832

5. Nellen JF, Wit FW, De Wolf F, Jurriaans S, Lange JM, Prins JM. Virologic and immunologic response to highly active antiretroviral therapy in indigenous and nonindigenous HIV-1-infected patients in the Netherlands. J Acquir Immune Defic Syndr. 2004;36(4):943–50. pmid:15220701

6. Nellen JF, Nieuwkerk PT, Burger DM, Wibaut M, Gras LA, Prins JM. Which method of adherence measurement is most suitable for daily use to predict virological failure among immigrant and non-immigrant HIV-1 infected patients? AIDS Care. 2009;21(7):842–50. pmid:20024740

7. van den Berg JB, Hak E, Vervoort SC, Hoepelman IM, Boucher CA, Schuurman R, et al. Increased risk of early virological failure in non-European HIV-1-infected patients in a Dutch cohort on highly active antiretroviral therapy. HIV Med. 2005;6(5):299–306. pmid:16156876

8. Monge S, Alejos B, Dronda F, Del Romero J, Iribarren JA, Pulido F, et al. Inequalities in HIV disease management and progression in migrants from Latin America and sub-Saharan Africa living in Spain. HIV Med. 2013;14(5):273–83. Epub 2012/11/23. pmid:23171059

9. Saracino A, Lorenzini P, Lo Caputo S, Girardi E, Castelli F, Bonfanti P, et al. Increased risk of virologic failure to the first antiretroviral regimen in HIV-infected migrants compared to natives: data from the ICONA cohort. Clin Microbiol Infect. 2016;22(3):288 e1–8.

10. Sumari-de Boer IM, Sprangers MA, Prins JM, Nieuwkerk PT. HIV stigma and depressive symptoms are related to adherence and virological response to antiretroviral treatment among immigrant and indigenous HIV infected patients. Aids Behav. 2012;16(6):1681–9. pmid:22198315

11. Been SK, van de Vijver DA, Nieuwkerk PT, Brito I, Stutterheim SE, Bos AE, et al. Risk Factors for Non-Adherence to cART in Immigrants with HIV Living in the Netherlands: Results from the ROtterdam ADherence (ROAD) Project. PLoS One. 2016;11(10):e0162800. pmid:27706251

12. Fox MP, Rosen S. Retention of Adult Patients on Antiretroviral Therapy in Low- and Middle-Income Countries: Systematic Review and Meta-analysis 2008–2013. J Acquir Immune Defic Syndr. 2015;69(1):98–108. pmid:25942461

13. de Boer IM, Prins JM, Sprangers MA, Nieuwkerk PT. Using different calculations of pharmacy refill adherence to predict virological failure among HIV-infected patients. J Acquir Immune Defic Syndr. 2010;55(5):635–40. pmid:21934556

14. Mekuria LA, Prins JM, Yalew AW, Sprangers MA, Nieuwkerk PT. Which adherence measure—self-report, clinician recorded or pharmacy refill—is best able to predict detectable viral load in a public ART programme without routine plasma viral load monitoring? Trop Med Int Health. 2016;21(7):856–69. pmid:27118068

15. Nieuwkerk PT, de Boer-van der Kolk IM, Prins JM, Locadia M, Sprangers MAG. Self-reported adherence is more predictive of virological treatment response among patients with a lower tendency towards socially desirable responding. Antiviral Therapy. 2010;15(6):913–6. pmid:20834104

16. Gordon LL, Gharibian D, Chong K, Chun H. Comparison of HIV Virologic Failure Rates Between Patients with Variable Adherence to Three Antiretroviral Regimen Types. AIDS Patient Care STDS. 2015;29(7):384–8. pmid:26114665

17. Berg KM, Wilson IB, Li X, Arnsten JH. Comparison of Antiretroviral Adherence Questions. Aids Behav. 2012;16(2):461–8. pmid:21181252

18. Nieuwkerk P, Gisolf E, Sprangers M, Danner S, Grp PS. Adherence over 48 weeks in an antiretroviral clinical trial: variable within patients, affected by toxicities and independently predictive of virological response. Antiviral Therapy. 2001;6(2):97–103. pmid:11491422

19. Nieuwkerk PT, Sprangers MAG, Burger DM, Hoetelmans RMW, Hugen PWH, Danner SA, et al. Limited patient adherence to highly active antiretroviral therapy for HIV-1 infection in an observational cohort study. Arch Intern Med. 2001;161(16):1962–8. pmid:11525698

20. Chesney MA, Ickovics JR, Chambers DB, Gifford AL, Neidig J, Zwickl B, et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG Adherence Instruments. Aids Care-Psychological and Socio-Medical Aspects of Aids/Hiv. 2000;12(3):255–66. pmid:10928201

21. Leierer G, Grabmeier-Pfistershammer K, Steuer A, Geit M, Sarcletti M, Haas B, et al. Factors Associated with Low-Level Viraemia and Virological Failure: Results from the Austrian HIV Cohort Study. PLoS One. 2015;10(11):e0142923. pmid:26566025

22. Sangeda RZ, Mosha F, Prosperi M, Aboud S, Vercauteren J, Camacho RJ, et al. Pharmacy refill adherence outperforms self-reported methods in predicting HIV therapy outcome in resource-limited settings. BMC Public Health. 2014;14:1035. pmid:25280535

23. Kabore L, Muntner P, Chamot E, Zinski A, Burkholder G, Mugavero MJ. Self-Report Measures in the Assessment of Antiretroviral Medication Adherence: Comparison with Medication Possession Ratio and HIV Viral Load. J Int Assoc Provid AIDS Care. 2015;14(2):156–62. pmid:25421930

24. Li JZ, Gallien S, Ribaudo H, Heisey A, Bangsberg DR, Kuritzkes DR. Incomplete adherence to antiretroviral therapy is associated with higher levels of residual HIV-1 viremia. Aids. 2014;28(2):181–6. pmid:24361679

25. Pasternak AO, de Bruin M, Jurriaans S, Bakker M, Berkhout B, Prins JM, et al. Modest nonadherence to antiretroviral therapy promotes residual HIV-1 replication in the absence of virological rebound in plasma. J Infect Dis. 2012;206(9):1443–52. pmid:22927449

26. Glass TR, De Geest S, Weber R, Vernazza PL, Rickenbach M, Furrer H, et al. Correlates of self-reported nonadherence to antiretroviral therapy in HIV-infected patients: the Swiss HIV Cohort Study. J Acquir Immune Defic Syndr. 2006;41(3):385–92. pmid:16540942

27. Saberi P, Neilands TB, Vittinghoff E, Johnson MO, Chesney M, Cohn SE. Barriers to antiretroviral therapy adherence and plasma HIV RNA suppression among AIDS clinical trials group study participants. AIDS Patient Care STDS. 2015;29(3):111–6. pmid:25615029