A hybrid type II effectiveness-implementation trial of a positive emotion regulation intervention among people living with HIV engaged in Ryan White Medical Case Management: protocol and design for the ORCHID study

Abstract

Background

The Ryan White Medical Case Management System, which serves more than half of people living with HIV (PLWH) in the USA, is an opportune setting for identifying and addressing depression among PLWH. A growing body of research suggests that interventions that promote positive emotion may lessen symptoms of depression and improve physical and psychological well-being among people experiencing a variety of health-related stress, including living with HIV. Research on how best to integrate standardized mental health screening and referral to evidence-based interventions in Ryan White Medical Case Management settings has the potential to improve the health and wellbeing of PLWH.

Methods

This mixed-methods study will enroll up to N = 300 Ryan White clients who screen positive for depressive symptoms in ORCHID (Optimizing Resilience and Coping with HIV through Internet Delivery), a web-based, self-guided positive emotion regulation intervention. The study will be conducted in 16 Ryan White Medical Case Management clinics in Chicago, IL. Following pre-implementation surveys and interviews with Medical Case Managers (MCMs) and Supervisors to develop an implementation facilitation strategy, we will conduct a hybrid type 2 implementation-effectiveness stepped wedge cluster randomized trial to iteratively improve the screening and referral process via interviews with MCMs in each wedge. We will test the effectiveness of ORCHID on depression and HIV care outcomes for PLWH enrolled in the program. RE-AIM is the implementation outcomes framework and the Consolidated Framework for Implementation Research is the implementation determinants framework.

Discussion

Study findings have the potential to improve mental health and substance use screening of Ryan White clients, decrease depression and improve HIV care outcomes, and inform the implementation of other evidence-based interventions in the Ryan White Medical Case Management System.

Trial registration

ClinicalTrials.gov NCT05123144. Trial registered 6/24/2021

Background

Depression is common among people living with HIV (PLWH) [1–3]. A meta-analysis of 118 studies from 2000 to 2018 demonstrated that the global estimated prevalence rate of depression was 31% (95% CI: 28–34%) among PLWH [4], with another meta-analysis estimating 41% [5]. Further, depression is underdiagnosed among PLWH underscoring the need for systematic screening in clinical settings [6, 7]. Depression is a barrier to optimal engagement in, and success with, HIV care [8–11]. For example, Gonzalez et al. [12] found that depression was significantly associated with HIV treatment nonadherence among 95 distinct samples of PLWH. Additionally, PLWH with untreated depression have a lower level of sustained viral suppression compared to those without depression or those receiving antidepressants [13].

Despite elevated rates of depression among PLWH and the documented deleterious associations with HIV progression [14], PLWH often lack access to adequate mental health screening [6] and treatment [7, 15]. In response, practitioners and researchers have called for universal mental health screening among PLWH and increased access to treatment options [16, 17]. Screening guidelines highlight the need for brief screening tools with high-burden populations, including PLWH experiencing trauma, violence, and other structural inequalities that can limit mental health screening and treatment access [18].

Once a mental health challenge is identified, there is an ethical imperative to provide evidence-based interventions to address mental health needs and well-being [16]. A growing body of evidence suggests that targeting the ability to experience and maintain positive emotion in the context of stress (rather than targeting the reduction of depression, stress, and negative emotion) is a promising approach to promoting physical and psychological adjustment when coping with a chronic illness [19, 20]. Grounded in theory [21, 22] and building on empirical links between positive affect and adaptive outcomes for people coping with significant stress, we developed an intervention that focuses on skills for increasing the frequency of positive affect for people coping with health-related or other life stress [23–28]. Our Positive Pathways to Health theoretical model [29] posits that increased positive affect has a range of proximal effects, such as providing a timeout from stress [30], prompting more adaptive coping strategies [21], broadened attention and cognition, and increased behavioral action tendencies [22], as well as reduced emotional reactivity to daily stress and strengthened social relationships, all of which lead to reduced depression [20, 29]. This reduction, in turn, predicts better physiological functioning (e.g., quicker autonomic recovery after a stressful event) [20, 31, 32] and greater adherence to recommended health behaviors [33–35], which ultimately lead to improved physical and psychological well-being. In concert with the rapidly growing literature on the social, cognitive, psychological, and health benefits of positive affect, our theoretical model argues strongly for programs that increase positive affect as an approach for mitigating depression, stress, and distress, and subsequently increasing the likelihood of beneficial health behaviors, ultimately improving physical health.

Among PLWH, positive emotion, separately from depression, predicts a slower progression of disease [14, 36], improves medication adherence [33, 37], and is associated with a greater likelihood of attaining a suppressed viral load [38]. In a previous randomized controlled trial conducted by our group, we found that sexual minority men (SMM) who endorsed substance abuse randomly assigned to a positive emotion skills intervention had lower viral load, increased positive affect, and reduced drug use when compared to SMM in an attention-control condition [39]. A separate trial showed that online delivery of the intervention resulted in decreased depression in a general population sample with elevated depressive symptoms [40, 41]. Finally, pilot testing of ORCHID (Optimizing Resilience and Coping with HIV through Internet Delivery), our self-guided, online, positive emotion regulation intervention, demonstrated feasibility, acceptability, and preliminary efficacy among PLWH with co-occurring depression [42].

The present paper describes the protocol for a hybrid type II effectiveness-implementation stepped wedge cluster randomized trial (SW-CRT) [43] of a clinic-based behavioral health screening and referral to ORCHID, a 6-week online positive emotion regulation intervention for people living with co-morbid HIV and depression. The study aims are to: (1) determine the effects of ORCHID on the primary outcome of depression symptoms and the secondary outcomes of retention in care, adherence to antiretroviral therapies (ART), viral suppression, and substance use; and (2) evaluate and iteratively refine the implementation strategies to support widespread adoption of mental health screening and the ORCHID intervention in Ryan White Medical Case Management settings.

We will use two implementation frameworks, each with distinct purposes aligned with study aims. The Consolidated Framework for Implementation Research (CFIR), a meta-theoretical framework based on 19 implementation theories and models, will be the determinants framework [44] to help us understand factors that determine implementation effectiveness. CFIR focuses on five domains: (1) the characteristics of the intervention; (2) the inner setting, i.e., where the intervention is implemented; (3) the outer setting, i.e., factors exogenous to the implementation setting; (4) the individuals involved in implementing the intervention; and (5) the implementation process [44]. In addition, we will use RE-AIM [45] as the evaluation framework to structure the factors on which we evaluate the success of the program. Like CFIR [44], RE-AIM is a widely used implementation outcomes framework [46] that focuses on five domains: Reach, Effectiveness, Adoption, Implementation, and Maintenance. In applying both frameworks, we will conduct cross-sectional and longitudinal mixed-methods research with MCMs and other clinical leaders to evaluate key implementation parameters, identify relevant implementation facilitators and barriers, and adapt the implementation strategy over the course of the trial.

In preparation for the stepped wedge trial, we conducted a sequential, explanatory mixed-methods study with medical case managers (MCMs) and supervisors across the network of 16 participating clinics. Through surveys with MCMs and supervisors followed by in-depth interviews, we identified barriers and facilitators to implementation and developed a set of initial implementation strategies. Please see Stump et al. for a full discussion of the methods and results of the pre-implementation research [47].

Materials and methods

Setting

The study is taking place in 16 Ryan White Medical Case Management Clinics in Chicago, IL. All participating clinics are part of the Northeastern Illinois HIV/AIDS Case Management Collaborative (the Collaborative), which is coordinated by AIDS Foundation Chicago (AFC), a study partner. AFC has led and coordinated the Collaborative since 1989. Through the Collaborative, AFC offers central administration and coordination to Ryan White Parts A and B funded clinics, including (1) standardized policies and procedures across sites, (2) intake and needs assessments, (3) standardized training and technical assistance, and (4) direct data entry of client-level reporting. Clinics that provide Ryan White medical case management but are not part of the Collaborative were not included in the study.

Behavioral health screener

AFC developed a behavioral health screener that includes standardized assessments of depression symptoms [48], anxiety symptoms [49], post-traumatic stress symptoms [50], alcohol use [51], and substance abuse [52]. Clients who screen positive for depression symptoms, as measured by the PHQ-9, are eligible to be referred to ORCHID.

ORCHID

ORCHID is a 6-week self-guided online positive emotion regulation intervention for PLWH. Hosted and maintained on the BrightOutcome platform, ORCHID teaches and reinforces a set of skills week by week, including savoring positive events, gratitude, mindfulness, positive reappraisal, personal strengths, goal setting, and acts of kindness. These skills and sessions have been described in-depth elsewhere [28, 39, 42, 53] (see Table 1). The intervention begins with an overview of the philosophies and structure of the course and presents a new skill or set of skills each week. This content is accompanied by daily practice, such as mindfulness exercises and gratitude journaling, as well as an instrument for brief emotion reporting.

Table 1. Overview of ORCHID intervention sessions, goal, and home practice

Skills	Goals of session and rationale for inclusion	Home practice
Session 1: positive events, savoring, and gratitude	Goal: Recognize positive events and the associated positive emotion; practice ways to amplify the experience of positive events; and learning to practice gratitude. Rationale:Positive life events (Skill 1) are associated with increases in positive emotion [54, 55] and scheduling “pleasant events” is a central part of some types of therapy for depression [56, 57]. Capitalizing (Skill 2) is an expressive response to positive events and includes telling others about it, marking the occurrence in some way, or even thinking about the event again later on [58]. Gratitude (Skill 3). Gratitude is defined as a feeling of thankfulness and appreciation. The association between intentionally noting things for which one is grateful and increased well-being is well-supported empirically [59–61].	Noting a positive event each day and writing about it (capitalizing); starting a daily gratitude journal and daily emotion reports. The gratitude list home practice continues through the rest of the 5-week intervention period.
Session 2: mindfulness	Goal: Learn and practice the awareness and nonjudgment components of mindfulness. Rationale:Mindfulness (Skill 4). Mindfulness is defined as the ability to intentionally pay attention to and maintain non-judgmental awareness of one’s thoughts, feelings, and physical sensations in the present moment [62]. Trait and state mindfulness are associated with higher positive emotion and lower negative emotion [63] and interventions to increase mindfulness have been shown to increase positive emotion [64, 65].	Daily informal mindfulness activities, a 10-min formal breath awareness activity, continuing the gratitude journal, and daily emotion reports.
Session 3: positive reappraisal	Goal: Understanding positive reappraisal and the idea that different forms of positive reappraisal can all lead to increased positive emotion in the face of stress. Rationale:Positive Reappraisal (Skill 5). According to stress and coping theory [66], the extent to which an event is experienced as stressful depends on the individual’s appraisal of the event and their resources for responding. Positive reappraisal is a form of coping in which the significance of the event is reinterpreted in a more positive way. In the coping literature, positive reappraisal is one of the few ways of coping that is consistently associated with increased positive emotions [21, 67].	Reporting a relatively minor stressor each day, then listing ways in which the event can be positively reappraised. The daily formal mindfulness practice, gratitude journal, and the emotion reports continue.
Session 4: personal strengths, achievable goals	Goal: Participant lists his or her personal strengths and notes how they may have used these strengths recently; Understanding characteristics of attainable goals and setting some goals for the week. Rationale:Focusing on Personal Strengths (Skill 6). Focusing on one’s strengths is a form of self affirmation that is sometimes used as a positive emotion manipulation in laboratory studies and self affirmation is also associated with positive emotion after failure feedback [68]. Other social psychological research demonstrates that self-enhancing cognitions (thoughts about one’s positive qualities) are associated with better psychological adjustment to illness [69] and healthier biological profiles [70]. Attainable goals (Skill 7). Goal setting is common in health education and intervention programs [71]. Observational research on goals indicates that perceptions of goal progress are associated with greater life satisfaction and higher levels of positive emotion [72, 73], and pursuit of attainable goals (vs. more global distant goals) is associated with higher subjective well being [74].	Listing a strength each day and how it was “expressed” behaviorally, working toward one of the attainable goals, and noting progress each day. The 10-min mindful breathing, the gratitude journal, and the daily emotion reports continue.
Session 5: self-compassion	Goal: Learn about how to show compassion for oneself, especially in the context of living with HIV. Understand how self-compassion relates to the other positive emotion skills in ORCHID. Rationale: Self-compassion involves regarding oneself with kindness and non-judgment, especially in times of suffering [75]. Although self-compassion was not included as a separate skill in the early iterations of our PPI, we added explicit self-compassion content as it became clear that many of our stressed samples were particularly hard on themselves. There was a need for participants to be less critical of themselves and this self-compassion made it easier for participants to engage with the other skills in the program.	A self-compassion log tracking daily acts of self-compassion.

Stepped wedged cluster randomized trial (SW-CRT)

Data collection will follow a stepped-wedge cluster design to recruit up to 300 PLWH receiving care at 16 Ryan White Medical Case Management clinics in Chicago, IL, USA. Recruitment for the evaluation of ORCHID effectiveness will last from the beginning of each wedge through the duration of the study. The 16 Ryan White clinics with co-located medical services were grouped into three clusters for randomization to timing of implementation of mental health screening + referral to ORCHID. We randomized groups using a modified constrained randomization process for cluster-based design [76] to ensure balance according to 12 baseline-level criteria pre-specified before randomization. The criteria included size, location, and other factors likely to influence the success of implementation as elucidated in the surveys and focus groups in the pre-implementation work [47]. Using this method, practices were randomly assigned into one of three wedges, and then the balance across waves was checked for those 12 criteria. Using REDCap, we simulated 30,000 randomization schemes, and of those, 15 distinct schemes met balancing criteria; from those 15, one was randomly selected for use in the project.

Each wedge will last approximately 12 months. See Fig. 1. Cluster assignment was concealed to participating clinics up until the start of the trial with trial recruitment starting in Oct 2021.

Fig. 1 [Images not available. See PDF.]

Scheme for stepped wedge cluster randomized trial

Behavioral health screening

All PLWH over age 18 receiving care at one of the participating Ryan White clinics are offered a behavioral health screener at intake. Should a client score above the clinical threshold on any scale, they are scheduled to complete the questionnaire every 6 months [47]. If clients score below the threshold or decline the screener, they are offered the screener again after 12 months. The behavioral health screener is being rolled out as standard care across the full network and is used for other referrals, in addition to ORCHID.

ORCHID study participants and recruitment

Clients with a PHQ-9 [48] score ≥ 5 on the behavioral screener are eligible for referral to ORCHID. MCMs provide a link to the study website and the study phone number to eligible clients, who then complete an online screener in REDCap in English or Spanish prior to consent and participation which includes: previous participation in the study, zip code, and whether the client has daily access to the internet.

ORCHID study retention

We plan a multi-pronged approach to ensure acceptable rates of retention in the study. First, we will work with AFC to re-contact participants enrolled in ORCHID who are lost to follow-up. AFC oversees all the Ryan White MCMs and will attempt to recontact participants through them. Second, we will work with the Institute for Sexual and Gender Minority Health at Northwestern which has been successful in attaining high rates of retention (> 80%) for online HIV prevention interventions in high-risk populations [77]. Finally, we will use participant locator software such as Alumni Finder and Lexus Nexus to find hard-to-reach participants.

Potential Harms: The risks of participating in ORCHID are minimal. In past research, the research team has not observed any participants experiencing serious or lasting distress in response to similar interventions or assessments. The intervention has been user-tested to remove any material/content that might be upsetting or insensitive, to reduce the chances of using those questions in future versions. Therefore, the risk of discomfort is extremely low.

Nonetheless, ORCHID study staff who are interacting with participants or monitoring their responses to study procedures must be alert to indicators of elevated distress or possible suicidality. If an ORCHID study staff member observes a possible sign of significant distress and/or suicidal ideation, they will follow a standard protocol to record details of the interaction, working together with designated team members to evaluate the severity of the situation, generate a tailored response, and complete final responding and reporting.

Measures

Participants who enroll in ORCHID will complete self-report assessments at baseline, post intervention (approximately 8 weeks later), and 6 months, and 12 months post-intervention (See Fig. 2).

Fig. 2 [Images not available. See PDF.]

SPIRIT Figure for a hybrid type II effectiveness-implementation trial of a positive emotion regulation intervention. *This study used a Stepped-Wedge Design meaning that 3 different clusters of clinics received the intervention in 3 different consecutive years (Wedge 1–3) see Fig. 1 ** T₂ = immediately post intervention (8 weeks), T₃ = 6 months post intervention, T₄ = 12 months post intervention. *** These will be derived from health records in aggregate by clinic see Measures

The primary effectiveness outcome is depression, as measured by depressive symptoms with the PROMIS CAT Depression scale [53]. Secondary outcomes are retention in care and viral suppression based on data from clinic EHRs. Retention in care is operationalized as the percentage of patients living with HIV with at least 2 encounters (> 89 days apart) within 12 months divided by the total number of patients living with HIV who had at least one medical encounter within 12 months [78] and viral suppression is operationalized following recommendations from the Centers for Disease Control [79], i.e., ≤ 200 RNA copies/mL defined. Secondary outcomes also include self-report measures of ARV adherence and engagement in HIV care. We will assess adherence over the past 30 days using a visual analog scale which is strongly correlated with objective measures of adherence [80]. For each HIV medication prescribed, the participant is asked to indicate how much of each drug they have taken in the past 30 days from 0 to 100%. In addition, we will ask how many doses were missed in the last week [81]. All measures are shown in Table 2. We will also examine maintenance of effects at the clinic level by examining changes in clinic-level viral suppression and engagement in care at the beginning and ending of each wedge as well as through the end of the study period.

Table 2. List of measures by construct domain with references

	Full name of measure	Administered ata:	Construct
1	Demographics	Baseline (T1) only	Demographics
2	Technology Survey	Right before intervention content	Technology/usability
3	System Usability Scale [82]	T2	Technology/usability
4	Daily Emotion Check-in [83]	All timepoints (T1, T2, T3, T4); and daily during the 8-week intervention period	Psychological adjustment
5	Daily Inventory of Stressful Events (DISE) [84]	All timepoints (T1, T2, T3, T4); daily during the 8-week intervention period; daily during the 8-week emotion reporting control period	Psychological adjustment
6	PHQ-9 [48]	Initial Eligibility Survey	Psychological adjustment
7	PROMIS Bank v1.0-Depression Adaptive Test (CAT) [85]	All timepoints	Psychological adjustment
8	PROMIS Anxiety Bank v1.0 Adaptive Test [85]	All timepoints	Psychological adjustment
9	Profile of Emotional Competence Scale [86]	All timepoints	Psychological adjustment
10	Emotional Complexity Scale [87]	All timepoints	Psychological adjustment
11	PROMIS SF v1.1 – Global Health [53]	All timepoints	Psychological adjustment
12	PROMIS Bank v1.0 – Sleep Disturbance Adaptive Test (CAT) [88]	All timepoints	Psychological adjustment
13	PROMIS Bank v1.0 – Meaning and Purpose Adaptive Test (CAT) [89]	All timepoints	Psychological adjustment
14	Healthcare Utilization [90, 91]	All timepoints	Health behaviors
15	Perceived Stress Scale [92]	All timepoints	Psychological adjustment
16	PROMIS General Life Satisfaction (CAT) [93]	All timepoints	Psychological adjustment
17	Self-Compassion, short form [94]	All timepoints	Psychological adjustment
18	COMBO Health Behavior Measures [95]	All timepoints	Health behaviors
19	Five Facet Mindfulness Questionnaire [96]	All timepoints	Psychological adjustment
20	Co-morbidity check-list	All timepoints	Comorbidity
21	PROMIS Bank v.10 Positive Affect Adaptive Test (CAT) [97], NeuroQoL Positive Affect and Well-Being v1.0 [98]	All timepoints	Psychological adjustment
22	AUDIT [51]	All timepoints	Health behaviors
23	DAST-10 [52]	All timepoints	Health behaviors
24	Medication Adherence [79, 80]	All timepoints	Health behaviors
26	Other Prescription Medications	All timepoints	Health behaviors
27	Intrusive and Avoidant thoughts about HIV [99]	All timepoints	Psychological adjustment
28	HIV Stigma Scale [100] Berger HIV Stigma Scale [101]	All timepoints	Psychological adjustment
29	Recommendations & Skill Use	T2	Technology/usability
30	Positive Resonance of Social Interactions [102, 103]	All timepoints	Psychological adjustment
31	Viral suppression (≤ 200 RNA copies/mL) [78]	Derived from medical records	Viral suppression
32	Retention in Care [77]	Medical records	Retention in care

^aT1 = baseline, T2 = immediately post intervention, T3 = 6 months post intervention, T4 = 12 months post intervention

Effectiveness analysis

Effectiveness analyses will examine the effect of ORCHID on depression symptoms and key secondary outcomes, with analyses occurring at both the individual and clinic levels. In modeling individual depressive symptoms, we will utilize multilevel latent growth curve modeling [104] to estimate adjusted latent growth curves while accounting for nesting of observations both with clusters and individuals over time [104, 105]. Similarly, viral load will be evaluated using a piecewise multilevel growth model [106]. The primary analysis will utilize multiple imputation leveraging baseline data to estimate missing values for those participants who are missing observations [107, 108]. In addition, a sensitivity analysis will compare this approach to listwise or pairwise deletion [109]. All analyses will be completed using Mplus [110]. At the clinic level, we will compare clinics where the intervention has been implemented (intervention clinics) to those where it has not yet been implemented (control clinics). We will use Wilcoxon signed-rank tests comparing rates of viral suppression by clinic before and after the introduction of the intervention [111, 112].

Power

Power analysis was conducted simulation in R [113] using variation in potential sample and effect sizes to estimate potential power to detect effects, assuming 16 clinics, a small intraclass correlation within sites (ICC = 0.05), and a substantial ICC within individuals (ICC = 0.30). We estimate that with 300 participants and a medium effect size for change in depression (d = 0.40), the study has sufficient power sufficient power (0.806) to detect a significant effect, with substantial power (0.950) to detect large effect sizes (d = 0.60) as found in our previous studies.

Implementation of evaluation procedures and participants

We will conduct an iterative mixed-methods evaluation of the implementation of the questionnaire and referral to ORCHID.

Qualitative interviews

We will conduct individual, in-depth interviews guided by the CFIR [44] with MCMs and supervisors in an active wedge at approximately 8 months post-implementation. The goal of the interviews is to understand barriers and facilitators to implementing the screener and referral to ORCHID to generate potential modifications to improve the implementation strategy package for the next cluster, as well as for future scalability and dissemination. MCMs and supervisors will be invited to participate in a confidential interview via email and provide informed consent via REDCap [114]. Interviews will be conducted via Zoom by trained members of the study team. Interviewers will take field notes [115] for each interview using a structured template designed to capture implementation facilitators, barriers, and insights, thereby capturing qualitative impressions and important implementation insights in real time [116]. All MCMs and supervisors who complete an interview will receive a $50 incentive in the form of a gift card. We also will track the response rate of MCMs and Supervisors who agree to participate, as well as reasons for refusal when provided.

Qualitative data analysis

Rapid qualitative methods are a pragmatic and robust approach to analyzing implementation data that can produce actionable and timeline insights [117, 118]. To start, we will analyze the field notes using the Stanford Lightning Report Method (SLRM) [119], a rapid qualitative synthesis method that organizes implementation data into three categories: (1) the plus: implementation facilitators or the factors that are working well, (2) the delta: barriers to implementation or the factors that need to change, and (3) insights: any implementation insights, recommendations or changes derived from the data or evaluators. Using the SLRM in real time will enable the team to quickly synthesize implementation barriers and facilitators and to identify potential changes to implementation strategies [116].

We will supplement this approach with Rapid Qualitative Analysis (RQA) [120, 121] of the transcripts. RQA is a rigorous, team-based approach to analyzing qualitative implementation data in order to produce actionable insights and outcomes. We will start with a structured template that follows the qualitative guides and facilitates rapid identification of key themes. Next, we will develop an analytic matrix that summarizes key findings and relevant quotes across transcripts. This approach, which we used in our pre-implementation research, has been found to be as rigorous as traditional qualitative analysis [118].

Implementation strategy adaptations

We will use the SLRM and RQA findings to identify the need for any adaptations or additions to the implementation strategy package used to support the referral and behavioral screener in each clinic wedge. All strategy adaptations will be recorded and reported in accordance to the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) [122]. For instance: initially, participants needed to be able to read English but in response to feedback from the initial set of clinics, we translated the platform and all materials into Spanish so that Spanish-speaking participants are now eligible.

Implementation outcome data tracking and data analysis

The majority of the data collected under RE-AIM is quantitative. With the exception of our effectiveness analyses, many of the analyses will be descriptive in order to provide quantitative perspectives on each outcome.

Reach

For mental health screening, we will measure the total number who are screened and referred. We will also compare characteristics (e.g., age, race/ethnicity, gender) of those who are screened to the population of patients in the clinic, and we will compare those who enroll in ORCHID to those who screened with elevated depression but did not enroll. We will also track the characteristics of those who refuse enrollment and those who withdraw or are lost to follow-up. We will use t-tests or chi-square tests to compare differences between targeted populations and those who enroll, with a Holm-modified Bonferroni correction to control for experiment-wise error rates, which minimizes both types I and II error rates [123].

Adoption

AFC will track the characteristics of clients who have appointments during the study period, those who are offered the behavioral health screener (MCM adoption), those who accept the Behavioral Health Screener (acceptability), and those who are eligible for and referred to (ORCHID staff adoption) and enrolled in ORCHID (client adoption). From these data, we can calculate adoption at the MCM, clinic, and client levels.

Implementation

Implementation includes the fidelity to the protocol delivery. With an online intervention such as ORCHID, delivery is highly consistent but we will track other aspects of implementation including indicators of fidelity, such as the percentage of ORCHID participants who complete the intervention sessions and the associated home practice (all data is tracked on the BrightOutcome platform).

At the study end, MCMs will be asked to complete a survey assessing acceptability, feasibility, and appropriateness of the behavioral health screener and referral to ORCHID using three brief scales that assess perspectives on intervention appeal and acceptability, how well the questionnaire and referral to ORCHID fits with MCM’s work, and the overall ease of implementing the questionnaire and referral process [124].

Maintenance

At the study end, MCMs will complete a survey assessing the potential for sustainability of the behavioral health screener and referral of ORCHID with the Clinical Sustainability Assessment Tool (CSAT), which assesses perceptions of organizational capacity to sustain the implementation of a new clinical practice in the future, including the organization’s ability to respond and adapt new practices in response to the dynamic contexts of any service environment [125]. The CSAT consists of 7 subscales with five items each that measure perceptions of (1) staff and leadership engagement, (2) key stakeholder engagement, (3) organizational readiness, (4) workflow integration, (5) implementation and training, (6) monitoring and evaluation, and (7) outcomes and effectiveness [124]. All items are assessed on a 1 = to little or no extent to 7 = a great extent [125].

In addition to the effectiveness outcome measured in the primary trial, clients will be asked the extent to which they continue to practice the skills they learned in ORCHID and any barriers to continued engagement.

Oversight and monitoring

The team overseeing and monitoring the trial consists of roughly 22 individuals who meet on a weekly basis and represent three major institutions: Northwestern University, University of Chicago, and AIDS Foundation of Chicago. The DSMB for this study is comprised of roughly 3 individuals who are external to the three aforementioned institutions. The DSMB meets at a minimum annually. The DSMB is independent of the sponsor and does not have competing interests. The charter for the DSMB is available on request. There is not a Stakeholder Public Involvement Group for this trial.

Data security

All quantitative data will be collected via secure HIPAA-compliant platforms such as REDCap. Data shared by AFC to academic partners will be de-identified and aggregated. Qualitative transcripts will be de-identified and all qualitative data will be stored on a secure HIPAA-compliant server.

Dissemination

Findings will be disseminated through academic journals, academic conferences, and community presentations (e.g., presentations to AFC and their constituents). Authorship will follow the guidelines of the International Committee of Medical Journal Editors guidelines [126].

Discussion

Persistent behavioral health inequities among PLWH highlight the need to increase evidence-based screening and referral in clinical care settings. The present study will be one of the first to examine the effectiveness of an evidence-based behavioral screener and referral to an online positive affect regulation intervention for PLWH who are receiving care in the Ryan White Medical Case Management System. Nationwide, the Ryan White Medical Case Management System serves over half of PLWH [127], with many encountering significant structural and psychosocial barriers to staying actively engaged in care and reaching viral suppression [128, 129]. Although positive affect interventions have evidence of efficacy, it is essential to address challenges to implementation, including the development of strategies that will enhance adoption into routine clinical or organizational practice. In addition, the ability to measure implementation outcomes and adapt existing or introduce new strategies to address barriers is important learning that can support scale-up to similar and new settings.

Using a type II hybrid effectiveness-implementation design [130] to scale out promising interventions for PLWH and depression is critical. Although numerous HIV interventions have demonstrated efficacy, few are widely disseminated and implemented [131]. In light of the ambitious timelines of the EHE plan [54, 132], we will use a hybrid type II effectiveness-implementation design [130] to yield simultaneous data on internal and external validity for screening and addressing depression among PWH receiving medical case management through Ryan White Parts A and B funded clinics in Chicago [46]. Compared to the typical decades-long approach that goes from efficacy to effectiveness to implementation, a hybrid type II design [130] will accelerate the time between research discovery and routine uptake, thereby maximizing the impact on Continuum of Care outcomes for PLWH and depression.

Trial status

Date of first recruitment: 10/1/2021.

Approximate date of completion: 6/30/2025.

Protocol Version: Version 15 (Approved 11/3/2023).

Acknowledgements

N/A.

Authors’ contributions

Casey D. Xavier Hall: conceptualization, coordination, drafting, editing. Kristen Ethier: conceptualization, drafting, editing. Peter Cummings: conceptualization, drafting, editing. Angela Freeman: conceptualization, drafting, editing. Katrin Bovbjerg: conceptualization, drafting, editing. Jacqueline Bannon: conceptualization, editing. Andrea Dakin: conceptualization, editing. Fay Abujado: conceptualization, editing. Nora Bouacha: conceptualization, editing. Devan Derricotte: conceptualization, editing. Lakethia Patterson: conceptualization, editing. Lisa R Hirschhorn: conceptualization, drafting, editing. Alida Bouris: conceptualization, drafting, editing, principal investigator. Judith T. Moskowitz: conceptualization, drafting, editing, principal investigator.

Funding

This trial was supported by a grant from the National Institute of Mental Health (R01MH124632; Moskowitz & Bouris). This project also was supported by a grant from the Third Coast Center for AIDS Research (CFAR), an NIH-funded program (P30 AI117943) supported by the following NIH co-funding and participating Institutes and Centers: NIAID, NCI, NICHD, NHLBI, NIDA, NIA, NIDDK, NIGMS, NIMH, NIMHD, NINR, NIDCR. Dr. Xavier Hall’s time was supported by a grant from the National Heart, Lung, and Blood Institute (1L60HL170367-01; PI: Xavier Hall). The content is solely the responsibility of the authors and does not necessarily reflect the official views of the funders.

Availability of data and materials

Data will be available upon request.

Declarations

Ethics approval and consent to participate

This study was approved by the Northwestern University Institutional Review Board (STU00209131).

Consent to publication

All participants consented to deidentified and aggregated data being used for publication.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Nanni, MG et al. Depression in HIV infected patients: a review. Curr Psychiatry Rep; 2015; 17, pp. 1-11. [DOI: https://dx.doi.org/10.1007/s11920-014-0530-4]

2. Do, AN et al. Excess burden of depression among HIV-infected persons receiving medical care in the United States: data from the medical monitoring project and the behavioral risk factor surveillance system. PLoS ONE; 2014; 9, 3 e92842. [DOI: https://dx.doi.org/10.1371/journal.pone.0092842] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24663122][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3963963]

3. Glynn, TR et al. High levels of syndemics and their association with adherence, viral non-suppression, and biobehavioral transmission risk in Miami, a US city with an HIV/AIDS epidemic. AIDS Behav; 2019; 23, pp. 2956-2965. [DOI: https://dx.doi.org/10.1007/s10461-019-02619-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31392443][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6803095]

4. Rezaei, S et al. Global prevalence of depression in HIV/AIDS: a systematic review and meta-analysis. BMJ Support Palliat Care; 2019; 9, 4 pp. 404-412. [DOI: https://dx.doi.org/10.1136/bmjspcare-2019-001952] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31537580]

5. Tao, J; Vermund, SH; Qian, H-Z. Association between depression and antiretroviral therapy use among people living with HIV: a meta-analysis. AIDS Behav; 2018; 22, pp. 1542-1550. [DOI: https://dx.doi.org/10.1007/s10461-017-1776-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28439754][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7942230]

6. Kilbourne, AM et al. Underdiagnosis of depression in HIV. J Gen Intern Med; 2003; 18, 6 pp. 450-460. [DOI: https://dx.doi.org/10.1046/j.1525-1497.2003.20938.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12823652][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1494868]

7. Conteh, NK; Latona, A; Mahomed, O. Mapping the effectiveness of integrating mental health in HIV programs: a scoping review. BMC Health Serv Res; 2023; 23, 1 pp. 1-14. [DOI: https://dx.doi.org/10.1186/s12913-023-09359-x]

8. Bhatia, R et al. Persons newly diagnosed with HIV infection are at high risk for depression and poor linkage to care: results from the steps study. AIDS Behav; 2011; 15, pp. 1161-1170. [DOI: https://dx.doi.org/10.1007/s10461-010-9778-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20711651][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3029485]

9. Ickovics, JR et al. Mortality, CD4 cell count decline, and depressive symptoms among HIV-seropositive women: longitudinal analysis from the HIV epidemiology research study. JAMA; 2001; 285, 11 pp. 1466-1474.[COI: 1:STN:280:DC%2BD3M7osFCmtA%3D%3D] [DOI: https://dx.doi.org/10.1001/jama.285.11.1466] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11255423]

10. Leserman, J et al. Relation of lifetime trauma and depressive symptoms to mortality in HIV. Am J Psychiatry; 2007; 164, 11 pp. 1707-1713. [DOI: https://dx.doi.org/10.1176/appi.ajp.2007.06111775] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17974936]

11. Mayne, TJ et al. Depressive affect and survival among gay and bisexual men infected with HIV. Arch Intern Med; 1996; 156, 19 pp. 2233-2238.[COI: 1:STN:280:DyaK2s%2FktVGisg%3D%3D] [DOI: https://dx.doi.org/10.1001/archinte.1996.00440180095012] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/8885823]

12. Gonzalez, JS et al. Depression and HIV/AIDS treatment nonadherence: a review and meta-analysis. J Acquir Immune Defic Syndr; 2011; 58, 2 pp. 181-7. [DOI: https://dx.doi.org/10.1097/QAI.0B013E31822D490A] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21857529]

13. Gokhale, RH et al. Depression prevalence, antidepressant treatment status, and association with sustained HIV viral suppression among adults living with HIV in care in the United States, 2009–2014. AIDS Behav; 2019; 23, pp. 3452-3459. [DOI: https://dx.doi.org/10.1007/s10461-019-02613-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31367965]

14. Ironson, G et al. Psychosocial and neurohormonal predictors of HIV disease progression (CD4 cells and viral load): a 4 year prospective study. AIDS Behav; 2015; 19, pp. 1388-1397.[COI: 1:STN:280:DC%2BC2M7ivFajtw%3D%3D] [DOI: https://dx.doi.org/10.1007/s10461-014-0877-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25234251][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4465405]

15. Cholera, R et al. Mind the gap: gaps in antidepressant treatment, treatment adjustments, and outcomes among patients in routine HIV care in a multisite US Clinical Cohort. PLoS ONE; 2017; 12, 1 e0166435. [DOI: https://dx.doi.org/10.1371/journal.pone.0166435] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28125593][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5268441]

16. Remien, RH et al. Mental health and HIV/AIDS: the need for an integrated response. AIDS (London, England); 2019; 33, 9 1411. [DOI: https://dx.doi.org/10.1097/QAD.0000000000002227] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30950883]

17. Organization, WH. Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. Recommendations for a public health approach ‐ Second edition; 2016; Geneva, World Health Organization:

18. Remien, RH et al. Integrating mental health into HIV prevention and care: a call to action. J Int AIDS Soc; 2021; 24, Suppl 2 e25748. [DOI: https://dx.doi.org/10.1002/jia2.25748] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34164925][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8222846]

19. Hernandez, R et al. Psychological well-being and physical health: associations, mechanisms, and future directions. Emot Rev; 2018; 10, 1 pp. 18-29. [DOI: https://dx.doi.org/10.1177/1754073917697824] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36650890]

20. Pressman, SD; Jenkins, BN; Moskowitz, JT. Positive affect and health: what do we know and where next should we go?. Annu Rev Psychol; 2019; 70, 1 pp. 627-650. [DOI: https://dx.doi.org/10.1146/annurev-psych-010418-102955] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30260746]

21. Folkman, S. Positive psychological states and coping with severe stress. Soc Sci Med; 1997; 45, pp. 1207-1221.[COI: 1:STN:280:DyaK1c%2FitVyhug%3D%3D] [DOI: https://dx.doi.org/10.1016/S0277-9536(97)00040-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/9381234]

22. Fredrickson, BL. What good are positive emotions?. Rev Gen Psychol; 1998; 2, pp. 300-319. [DOI: https://dx.doi.org/10.1037/1089-2680.2.3.300] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21850154][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3156001]

23. Cheung EO, Cohn MA, Dunn LB, Melisko ME, Morgan S, Penedo FJ, Salsman JM, Shumay DM, Moskowitz JT. A Randomized Pilot Trial of a Positive Affect Skill Intervention (Lessons in Linking Affect and Coping) for Women with Metastatic Breast Cancer. Psycho-Oncology. 2017;26(12):2101–8. https://doi.org/10.1002/pon.4312.

24. Moskowitz J, et al. Randomized controlled trial of a positive affect intervention to reduce stress in people newly diagnosed with HIV; protocol and design for the IRISS study. Open Access J Clin Trials. 2014;2014(6):85–100.

25. Moskowitz, JT et al. A positive affect intervention for people experiencing health-related stress: development and non-randomized pilot test. J Health Psychol; 2012; 17, 5 pp. 677-693. [DOI: https://dx.doi.org/10.1177/1359105311425275]

26. Cohn, MA et al. An online positive affect skills intervention reduces depression in adults with type 2 diabetes. J Posit Psychol; 2014; 9, 6 pp. 523-534. [DOI: https://dx.doi.org/10.1080/17439760.2014.920410] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25214877][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4157680]

27. Dowling, GA et al. Life enhancing activities for family caregivers of people with frontotemporal dementia. Alzheimer Dis Assoc Disord; 2014; 28, 2 pp. 175-181. [DOI: https://dx.doi.org/10.1097/WAD.0b013e3182a6b905] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24113564]

28. Moskowitz, JT et al. Randomized controlled trial of a positive affect intervention for people newly diagnosed with HIV. J Consult Clin Psychol; 2017; 85, 5 pp. 409-423. [DOI: https://dx.doi.org/10.1037/ccp0000188] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28333512][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5398944]

29. Moskowitz, JT; Addington, EA; Cheung, EO. Positive pathways to health. Gen Hosp Psychiatry; 2019; 61, pp. 136-138. [DOI: https://dx.doi.org/10.1016/j.genhosppsych.2019.11.001] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31757566]

30. Lazarus, RS; Kanner, AD; Folkman, S. Plutchik, R; Kellerman, H. Emotions: A cognitive-phenomenological analysis. Theories of emotion; 1980; New York, Academic Press: pp. 189-217. [DOI: https://dx.doi.org/10.1016/B978-0-12-558701-3.50014-4]

31. Fredrickson, BL; Levenson, RW. Positive emotions speed recovery from the cardiovascular sequelae of negative emotions. Cogn Emot; 1998; 12, pp. 191-220. [DOI: https://dx.doi.org/10.1080/026999398379718] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21852890][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3156608]

32. Pressman, SD; Cohen, S. Does positive affect influence health?. Psychol Bull; 2005; 131, 6 pp. 925-71. [DOI: https://dx.doi.org/10.1037/0033-2909.131.6.925] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16351329]

33. Carrico, AW; Moskowitz, JT. Positive affect promotes engagement in care after HIV diagnosis. Health Psychol; 2014; 33, 7 686. [DOI: https://dx.doi.org/10.1037/hea0000011] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24245846]

34. Bassett, SM et al. Positive affect and medication adherence in chronic conditions: a systematic review. Health Psychol; 2019; 38, 11 960. [DOI: https://dx.doi.org/10.1037/hea0000778] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31368717][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6800780]

35. Hoogwegt, MT et al. Exercise mediates the association between positive affect and 5-year mortality in patients with ischemic heart disease. Circulation; 2013; 6, pp. 559-66. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24021694]

36. Ironson, GH. Do positive psychosocial factors predict disease progression in HIV-1? A review of the evidence. Psychosom Med; 2008; 70, 5 546. [DOI: https://dx.doi.org/10.1097/PSY.0b013e318177216c] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18541905][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2614870]

37. Gonzalez, JS et al. Social support, positive states of mind, and HIV treatment adherence in men and women living with HIV/AIDS. Health Psychol; 2004; 23, 4 413. [DOI: https://dx.doi.org/10.1037/0278-6133.23.4.413] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/15264978]

38. Wilson, TE et al. Positive affect and its association with viral control among women with HIV infection. Health Psychol; 2017; 36, 1 91. [DOI: https://dx.doi.org/10.1037/hea0000382] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27685456]

39. Carrico, AW et al. Randomized controlled trial of a positive affect intervention to reduce HIV viral load among sexual minority men who use methamphetamine. J Int AIDS Soc; 2019; 22, 12 e25436. [DOI: https://dx.doi.org/10.1002/jia2.25436] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31860172][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6924317]

40. Addington, EL et al. The MARIGOLD study: feasibility and enhancement of an online intervention to improve emotion regulation in people with elevated depressive symptoms. J Affect Disord; 2019; 257, pp. 352-364. [DOI: https://dx.doi.org/10.1016/j.jad.2019.07.049] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31302525][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6711819]

41. Cheung, EO et al. A self-paced, web-based, positive emotion skills intervention for reducing symptoms of depression: protocol for development and pilot testing of MARIGOLD. JMIR Res Protoc; 2018; 7, 6 e10494. [DOI: https://dx.doi.org/10.2196/10494] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29871853][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6008514]

42. Bassett, S et al. Feasibility and acceptability of an online positive affect intervention for those living with comorbid HIV depression. AIDS Behav; 2019; 23, pp. 753-764.[COI: 1:STN:280:DC%2BB3cjnt1ajtw%3D%3D] [DOI: https://dx.doi.org/10.1007/s10461-019-02412-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30701389]

43. Hemming, K et al. Stepped wedge cluster randomized trials are efficient and provide a method of evaluation without which some interventions would not be evaluated. J Clin Epidemiol; 2013; 66, 9 pp. 1058-1059. [DOI: https://dx.doi.org/10.1016/j.jclinepi.2012.12.020] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23849737]

44. Damschroder, LJ et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci; 2009; 4, 1 50. [DOI: https://dx.doi.org/10.1186/1748-5908-4-50] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19664226][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2736161]

45. Glasgow, RE; Vogt, TM; Boles, SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health; 1999; 89, 9 pp. 1322-1327.[COI: 1:STN:280:DyaK1MvgtlOqtw%3D%3D] [DOI: https://dx.doi.org/10.2105/AJPH.89.9.1322] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/10474547][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1508772]

46. Glasgow, RE et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a twenty-year review. Front Public Health; 2019; 7, 64. [DOI: https://dx.doi.org/10.3389/fpubh.2019.00064] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30984733][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6450067]

47. Stump, TK et al. Development of an implementation facilitation strategy to link mental health screening and eHealth intervention for clients in Ryan white-funded clinics in Chicago. J Acquir Immune Defic Syndr; 2022; 90, 1 pp. S197-S205. [DOI: https://dx.doi.org/10.1097/QAI.0000000000002980] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35703772][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9204837]

48. Kroenke, K; Spitzer, RL; Williams, JB. The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med; 2001; 16, 9 pp. 606-613.[COI: 1:STN:280:DC%2BD3MrhtVWnsQ%3D%3D] [DOI: https://dx.doi.org/10.1046/j.1525-1497.2001.016009606.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11556941][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1495268]

49. Kroenke, K et al. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med; 2007; 146, 5 pp. 317-325. [DOI: https://dx.doi.org/10.7326/0003-4819-146-5-200703060-00004] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17339617]

50. Prins, A et al. The primary care PTSD screen for DSM-5 (PC-PTSD-5): development and evaluation within a veteran primary care sample. J Gen Intern Med; 2016; 31, 10 pp. 1206-1211. [DOI: https://dx.doi.org/10.1007/s11606-016-3703-5] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27170304][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5023594]

51. Bush, K et al. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Arch Intern Med; 1998; 158, 16 pp. 1789-1795.[COI: 1:STN:280:DyaK1cvgvV2rug%3D%3D] [DOI: https://dx.doi.org/10.1001/archinte.158.16.1789] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/9738608]

52. Cocco, KM; Carey, KB. Psychometric properties of the Drug Abuse Screening Test in psychiatric outpatients. Psychol Assess; 1998; 10, 4 408. [DOI: https://dx.doi.org/10.1037/1040-3590.10.4.408]

53. Hays, RD et al. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual Life Res; 2009; 18, pp. 873-880. [DOI: https://dx.doi.org/10.1007/s11136-009-9496-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19543809][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2724630]

54. Services, D.o.H.a.H. Ending the HIV epidemic: a plan for America. 2019. Available from: https://www.hhs.gov/sites/default/files/ending-the-hiv-epidemic-fact-sheet.pdf.

55. Murrell, SA; Norris, FH. Resources, life events, and changes in positive affect and depression in older adults. Am J Community Psychol; 1984; 12, 4 pp. 445-464.[COI: 1:STN:280:DyaL2M%2FitVSlug%3D%3D] [DOI: https://dx.doi.org/10.1007/BF00896505] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/6486109]

56. Zautra, AJ; Reich, JW. Life events and perceptions of life quality: developments in a two-factor approach. J Community Psychol; 1983; 11, pp. 121-132. [DOI: https://dx.doi.org/10.1002/1520-6629(198304)11:2<121::AID-JCOP2290110206>3.0.CO;2-V]

57. Krause, N. Positive life events and depressive symptoms in older adults. Behav Med; 1998; 14, pp. 101-112. [DOI: https://dx.doi.org/10.1080/08964289.1988.9935131]

58. Lewinsohn, PM; Hoberman, HM; Clarke, GN. The coping with depression course: review and future directions. Can J Behav Sci; 1989; 21, pp. 470-493. [DOI: https://dx.doi.org/10.1037/h0079846]

59. Langston, CA. Capitalizing on and coping with daily-life events: expressive responses to positive events. J Pers Soc Psychol; 1994; 67, pp. 1112-2112. [DOI: https://dx.doi.org/10.1037/0022-3514.67.6.1112]

60. Emmons, RA. Thanks! how the new science of gratitude can make you happier; 2007; New York, Houghton Mifflin:

61. Emmons, RA; McCullough, ME. Counting blessings versus burdens: an experimental investigation of gratitude and subjective well-being in daily life. J Pers Soc Psychol; 2003; 84, pp. 377-389. [DOI: https://dx.doi.org/10.1037/0022-3514.84.2.377] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12585811]

62. Kashdan, TB; Uswatte, G; Julian, T. Gratitude and hedonic and eudaimonic well-being in Vietnam war veterans. Behav Res Ther; 2006; 44, pp. 177-199. [DOI: https://dx.doi.org/10.1016/j.brat.2005.01.005] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16389060]

63. Kabat-Zinn, J. Mindfulness-based interventions in context: past, present, and future. Clin Psychol Sci Pract; 2003; 10, pp. 144-156. [DOI: https://dx.doi.org/10.1093/clipsy.bpg016]

64. Brown, KW; Ryan, RM. The benefits of being present: mindfulness and its role in psychological well-being. J Pers Soc Psychol; 2003; 84, pp. 822-848. [DOI: https://dx.doi.org/10.1037/0022-3514.84.4.822] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12703651]

65. Fredrickson, BL et al. Open hearts build lives: positive emotions, induced through meditation, build consequential personal resources. J Pers Soc Psychol; 2008; 95, pp. 1045-1062. [DOI: https://dx.doi.org/10.1037/a0013262] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18954193][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3156028]

66. Grossman, P et al. Mindfulness training as an intervention for fibromyalgia: evidence of postintervention and 3-year follow-up benefits in well-being. Psychother Psychosom; 2007; 76, pp. 226-233. [DOI: https://dx.doi.org/10.1159/000101501] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17570961]

67. Lazarus, RS; Folkman, S. Stress, appraisal, and coping; 1984; New York, Springer:

68. Sears, SR; Stanton, AL; Danoff-Burg, S. The yellow brick road and the emerald city: Benefit finding, positive reappraisal coping and posttraumatic growth in women with early-stage breast cancer. Health Psychol; 2003; 22, 5 pp. 487-497. [DOI: https://dx.doi.org/10.1037/0278-6133.22.5.487] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/14570532]

69. Koole, SL et al. The cessation of rumination through self-affirmation. J Pers Soc Psychol; 1999; 77, pp. 111-125. [DOI: https://dx.doi.org/10.1037/0022-3514.77.1.111]

70. Taylor, SE et al. Maintaining positive illusions in the face of negative information: getting the facts without letting them get to you. J Soc Clin Psychol; 1989; 8, 2 pp. 114-129. [DOI: https://dx.doi.org/10.1521/jscp.1989.8.2.114]

71. Taylor, SE et al. Are self-enhancing cognitions associated with healthy or unhealthy biological profiles?. J Pers Soc Psychol; 2003; 85, pp. 605-615. [DOI: https://dx.doi.org/10.1037/0022-3514.85.4.605] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/14561115]

72. Strecher, VJ et al. Goal setting as a strategy for health behavior change. Health Educ Q; 1995; 22, pp. 190-200.[COI: 1:STN:280:DyaK2Mzksl2qsg%3D%3D] [DOI: https://dx.doi.org/10.1177/109019819502200207] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/7622387]

73. Carver, CS; Scheier, MF. Origins and functions of positive and negative affect: a control process view. Psychol Rev; 1990; 97, pp. 19-35. [DOI: https://dx.doi.org/10.1037/0033-295X.97.1.19]

74. Lent, RW et al. Social cognitive predictors of domain and life satisfaction: exploring the theoretical precursors of subjective well-being. J Consult Clin Psychol; 2005; 52, pp. 429-442.

75. Emmons, RA. Abstract versus concrete goals: personal striving level, physical illness, and psychological well-being. J Pers Soc Psychol; 1992; 62, pp. 292-300.[COI: 1:STN:280:DyaK383htFGjug%3D%3D] [DOI: https://dx.doi.org/10.1037/0022-3514.62.2.292] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/1556661]

76. Nietert, PJ; Jenkins, RG; Nemeth, LS; Ornstein, SM. An application of a modified constrained randomization process to a practice-based cluster randomized trial to improve colorectal cancer screening. Contemp Clin Trials; 2009; 30, 2 pp. 129-132. [DOI: https://dx.doi.org/10.1016/j.cct.2008.10.002] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18977314]

77. Mustanski, B et al. Feasibility, acceptability, and preliminary efficacy of an online HIV prevention program for diverse young men who have sex with men: the keep it up! intervention. AIDS Behav; 2013; 17, pp. 2999-3012. [DOI: https://dx.doi.org/10.1007/s10461-013-0507-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23673793]

78. Administration HRS. Performance measure: annual retention in care. 2023. Available from: https://ryanwhite.hrsa.gov/grants/performance-measure-portfolio/core-measures/annual-retention-in-care. [cited 2024].

79. Prevention, C.f.D.C.a. Understanding the HIV Care Continuum. 2016. https://www.cdc.gov/hiv/pdf/library/factsheets/cdc-hiv-care-continuum.pdf.

80. Walsh, JC; Mandalia, S; Gazzard, BG. Responses to a 1 month self-report on adherence to antiretroviral therapy are consistent with electronic data and virological treatment outcome. AIDS; 2002; 16, 2 pp. 269-77. [DOI: https://dx.doi.org/10.1097/00002030-200201250-00017] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11807312]

81. Simoni, JM et al. Self-report measures of antiretroviral therapy adherence: a review with recommendations for HIV research and clinic management. AIDS Behav; 2006; 10, pp. 227-245. [DOI: https://dx.doi.org/10.1007/s10461-006-9078-6] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16783535][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4083461]

82. Ferrari, M et al. Self-compassion interventions and psychosocial outcomes: a meta-analysis of RCTs. Mindfulness; 2019; 10, pp. 1455-1473. [DOI: https://dx.doi.org/10.1007/s12671-019-01134-6]

83. Brooke J. Sus: a “quick and dirty’usability. Usability evaluation in industry. Open Access J Clin Trials. 1996;189(3):189–94.

84. Izard CE. Human emotions. New York: Springer Science & Business Media; 2013.

85. Almeida, DM; Wethington, E; Kessler, RC. The daily inventory of stressful events: an interview-based approach for measuring daily stressors. Assessment; 2002; 9, 1 pp. 41-55. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11911234]

86. Pilkonis, PA et al. Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): depression, anxiety, and anger. Assessment; 2011; 18, 3 pp. 263-283. [DOI: https://dx.doi.org/10.1177/1073191111411667] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21697139][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3153635]

87. Brasseur, S et al. The profile of emotional competence (PEC): Development and validation of a self-reported measure that fits dimensions of emotional competence theory. PLoS ONE; 2013; 8, 5 e62635.[COI: 1:CAS:528:DC%2BC3sXns1Ortbc%3D] [DOI: https://dx.doi.org/10.1371/journal.pone.0062635] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23671616][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3646043]

88. Kang, SM; Shaver, PR. Individual differences in emotional complexity: their psychological implications. J Pers; 2004; 72, 4 pp. 687-726. [DOI: https://dx.doi.org/10.1111/j.0022-3506.2004.00277.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/15210014]

89. van Kooten, JA et al. Validation of the PROMIS sleep disturbance and sleep-related impairment item banks in Dutch adolescents. Qual Life Res; 2018; 27, pp. 1911-1920. [DOI: https://dx.doi.org/10.1007/s11136-018-1856-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29663257][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5997729]

90. Salsman, JM et al. Assessing meaning & purpose in life: development and validation of an item bank and short forms for the NIH PROMIS®. Qual Life Res; 2020; 29, pp. 2299-2310. [DOI: https://dx.doi.org/10.1007/s11136-020-02489-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32306302][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7367748]

91. Ritter, PL et al. Self-reports of health care utilization compared to provider records. J Clin Epidemiol; 2001; 54, 2 pp. 136-141.[COI: 1:STN:280:DC%2BD3M7ls12isg%3D%3D] [DOI: https://dx.doi.org/10.1016/S0895-4356(00)00261-4] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11166528]

92. Lorig K. Outcome measures for health education and other health care interventions. New York: Sage; 1996.

93. Warttig, SL et al. New, normative, English-sample data for the short form perceived stress scale (PSS-4). J Health Psychol; 2013; 18, 12 pp. 1617-1628. [DOI: https://dx.doi.org/10.1177/1359105313508346] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24155195]

94. Salsman, JM et al. Refining and supplementing candidate measures of psychological well-being for the NIH PROMIS®: qualitative results from a mixed cancer sample. Qual Life Res; 2018; 27, pp. 2471-2476. [DOI: https://dx.doi.org/10.1007/s11136-018-1896-2] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29926344][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6677243]

95. Neff, KD et al. The development and validation of the state self-compassion scale (long-and short form). Mindfulness; 2021; 12, pp. 121-140. [DOI: https://dx.doi.org/10.1007/s12671-020-01505-4]

96. Fernald, DH et al. Common measures, better outcomes (COMBO): a field test of brief health behavior measures in primary care. Am J Prev Med; 2008; 35, 5 pp. S414-S422. [DOI: https://dx.doi.org/10.1016/j.amepre.2008.08.006] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18929989]

97. Gu, J et al. Examining the factor structure of the 39-item and 15-item versions of the Five Facet Mindfulness Questionnaire before and after mindfulness-based cognitive therapy for people with recurrent depression. Psychol Assess; 2016; 28, 7 791. [DOI: https://dx.doi.org/10.1037/pas0000263] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27078186][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4928699]

98. Forrest, CB et al. Development and evaluation of the PROMIS® pediatric positive affect item bank, child-report and parent-proxy editions. J Happiness Stud; 2018; 19, pp. 699-718. [DOI: https://dx.doi.org/10.1007/s10902-016-9843-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29760578]

99. Salsman, JM et al. Development and validation of the positive affect and well-being scale for the neurology quality of life (Neuro-QOL) measurement system. Qual Life Res; 2013; 22, pp. 2569-2580. [DOI: https://dx.doi.org/10.1007/s11136-013-0382-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23526093]

100. Horowitz, M; Wilner, N; Alvarez, W. Impact of event scale: a measure of subjective stress. Psychosom Med; 1979; 41, 3 pp. 209-218.[COI: 1:STN:280:DyaE1M3kslGjug%3D%3D] [DOI: https://dx.doi.org/10.1097/00006842-197905000-00004] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/472086]

101. Sayles, JN et al. Development and psychometric assessment of a multidimensional measure of internalized HIV stigma in a sample of HIV-positive adults. AIDS Behav; 2008; 12, pp. 748-758. [DOI: https://dx.doi.org/10.1007/s10461-008-9375-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18389363][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2858334]

102. Fuster-RuizdeApodaca, MJ et al. Adaptation of the HIV stigma scale in Spaniards with HIV. Span J Psychol; 2015; 18, E66. [DOI: https://dx.doi.org/10.1017/S1138741615000694] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26369905]

103. West, TN et al. How the affective quality of social connections may contribute to public health: prosocial tendencies account for the links between positivity resonance and behaviors that reduce the spread of COVID-19. Affect Sci; 2021; 2, 3 pp. 241-261. [DOI: https://dx.doi.org/10.1007/s42761-021-00035-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33870213][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8041949]

104. Beard, E et al. Stepped wedge randomised controlled trials: systematic review of studies published between 2010 and 2014. Trials; 2015; 16, 1 pp. 1-14. [DOI: https://dx.doi.org/10.1186/s13063-015-0839-2]

105. Prost, A et al. Logistic, ethical, and political dimensions of stepped wedge trials: critical review and case studies. Trials; 2015; 16, 1 pp. 1-11. [DOI: https://dx.doi.org/10.1186/s13063-015-0837-4]

106. Kohli, N; Harring, JR. Modeling growth in latent variables using a piecewise function. Multivar Behav Res; 2013; 48, 3 pp. 370-397. [DOI: https://dx.doi.org/10.1080/00273171.2013.778191]

107. Li, J et al. Roles of self-stigma, social support, and positive and negative affects as determinants of depressive symptoms among HIV infected men who have sex with men in China. AIDS Behav; 2017; 21, pp. 261-273. [DOI: https://dx.doi.org/10.1007/s10461-016-1321-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26896120][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4992470]

108. Wohl, DA et al. A randomized controlled trial of an intervention to maintain suppression of HIV viremia following prison release: The imPACT trial-JAIDS Ms. no.: QAIV16986. J Acquir Immune Defic Syndr; 2017; 75, 1 81. [DOI: https://dx.doi.org/10.1097/QAI.0000000000001337] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28277487][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5443668]

109. Enders, CK; Bandalos, DL. The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Struct Equ Model; 2001; 8, 3 pp. 430-457. [DOI: https://dx.doi.org/10.1207/S15328007SEM0803_5]

110. Muthén L.K.a.B.O.M. Mplus user’s guide, 3rd ed. 2005.

111. Brown, CA; Lilford, RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol; 2006; 6, 1 pp. 1-9. [DOI: https://dx.doi.org/10.1186/1471-2288-6-54]

112. Woolson R. Wilcoxon signed‐rank test. 2007. p. 1–3.

113. Green, P; MacLeod, CJ. SIMR: an R package for power analysis of generalized linear mixed models by simulation. Methods Ecol Evol; 2016; 7, 4 pp. 493-498. [DOI: https://dx.doi.org/10.1111/2041-210X.12504]

114. Patridge, EF; Bardyn, TP. Research electronic data capture (REDCap). J Med Lib Assoc; 2018; 106, 1 142.

115. Phillippi, J; Lauderdale, J. A guide to field notes for qualitative research: context and conversation. Qual Health Res; 2018; 28, 3 pp. 381-388. [DOI: https://dx.doi.org/10.1177/1049732317697102] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29298584]

116. Brown-Johnson, C et al. The Stanford Lightning Report Method: a comparison of rapid qualitative synthesis results across four implementation evaluations. Learning Health Systems; 2020; 4, 2 e10210. [DOI: https://dx.doi.org/10.1002/lrh2.10210] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32313836]

117. Ridgeway, JL et al. Reducing burden and building goodwill for practice-embedded trials: results of rapid qualitative methods in the preimplementation phase of a community paramedic trial to reduce hospitalizations. J Clin Transl Sci; 2023; 7, 1 e61. [DOI: https://dx.doi.org/10.1017/cts.2023.18] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37008618][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10052418]

118. Gale, RC et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci; 2019; 14, 1 11. [DOI: https://dx.doi.org/10.1186/s13012-019-0853-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30709368][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6359833]

119. Brown-Johnson, C et al. The stanford lightning report method: a comparison of rapid qualitative synthesis results across four implementation evaluations. Learn Health Syst; 2020; 4, 2 e10210. [DOI: https://dx.doi.org/10.1002/lrh2.10210] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32313836]

120. Hamilton A. Qualitative methods in rapid turn-around health services research., V.H.S. Research, Editor. 2013: https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780.

121. Hamilton AB. Rapid qualitative analysis: updates/developments. V.H.S.R.a. Development, Editor. 2020. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=3846.

122. Miller, CJ et al. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci; 2021; 16, pp. 1-12. [DOI: https://dx.doi.org/10.1186/s13012-021-01105-3]

123. Jaccard, J; Guilamo-Ramos, V. Analysis of variance frameworks in clinical child and adolescent psychology: issues and recommendations. J Clin Child Adolesc Psychol; 2002; 31, 1 pp. 130-146. [DOI: https://dx.doi.org/10.1207/S15374424JCCP3101_15] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11845645]

124. Weiner, BJ et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci; 2017; 12, pp. 1-12. [DOI: https://dx.doi.org/10.1186/s13012-017-0635-3]

125. Malone, S et al. The clinical sustainability assessment tool: measuring organizational capacity to promote sustainability in healthcare. Implement Sci Commun; 2021; 2, pp. 1-12. [DOI: https://dx.doi.org/10.1186/s43058-021-00181-2]

126. Editors, I.C.o.M.J. Defining the Role of Authors and Contributors. 2024. Available from: https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html. [cited 2024].

127. Program, H.R.S.A.R.W.H.A. Program Parts & Initiatives. 2023. Available from: https://ryanwhite.hrsa.gov/about/parts-and-initiatives.

128. Park, E et al. Structural barriers to women’s sustained engagement in HIV care in southern California. AIDS Behav; 2020; 24, pp. 2966-2974. [DOI: https://dx.doi.org/10.1007/s10461-020-02847-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32323105][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7790164]

129. Marhefka, SL et al. Social determinants of potential eHealth engagement among people living with HIV receiving Ryan white case management: health equity implications from project TECH. AIDS Behav; 2020; 24, pp. 1463-1475. [DOI: https://dx.doi.org/10.1007/s10461-019-02723-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31828450]

130. Curran, GM et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care; 2012; 50, 3 217. [DOI: https://dx.doi.org/10.1097/MLR.0b013e3182408812] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22310560][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3731143]

131. Noar, SM. Computer technology-based interventions in HIV prevention: state of the evidence and future directions for research. AIDS Care; 2011; 23, 5 pp. 525-533. [DOI: https://dx.doi.org/10.1080/09540121.2010.516349] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21287420][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4516027]

132. Fauci, AS et al. Ending the HIV epidemic: a plan for the United States. JAMA; 2019; 321, 9 pp. 844-845. [DOI: https://dx.doi.org/10.1001/jama.2019.1343] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30730529]