Recruitment effort and costs from a multi-center randomized controlled trial for treating depression in type 2 diabetes

Abstract

Background

Participant recruitment for clinical trials is a significant challenge for the scientific research community. Federal funding agencies have made continuation of funding of clinical trials contingent on meeting recruitment targets. It is incumbent on investigators to carefully set study recruitment timelines and resource needs to meet those goals as required under current funding mechanisms. This paper highlights the cost, labor, and barriers to recruitment for Program ACTVE II, a successful multisite randomized controlled trial of behavioral treatments for depression in adults with type 2 diabetes, conducted in rural and urban settings in three states.

Methods

Quantitative and qualitative data on recruitment were gathered from study staff throughout the study recruitment period and were used to calculate costs and effort. The study utilized two main approaches to recruitment: (1) relying on potential participants to see ads in the community and call a toll-free number; and (2) direct phone calls to potential participants by study staff.

Results

Contact was attempted with 18,925 people to obtain the enrolled sample of 140. The cost of recruitment activities during the 4.5-year recruitment period totaled $190,056, an average cost of $1358 per enrolled participant. Qualitative evaluations identified multiple barriers to recruitment.

Conclusions

Recruitment for Program ACTIVE II exemplifies the magnitude of resources needed to reach recruitment targets in the current era. Continuous evaluation, flexibility, and adaptation are required on the part of investigators, community partners, and funding agencies to successfully reach high-risk populations in rural and urban areas.

Trial registration

ClinicalTrials.gov, NCT03371940. Registered on 13 December 2017.

Background

Randomized controlled trials (RCT) remain the gold standard for evaluating the effectiveness of interventions in healthcare research. Critical to the success of these trials is recruitment of study participants in order to have sufficient statistical power to detect the effects of interventions [1]. Recruitment is increasingly challenging for clinical trials across all domains of health research, particularly for studies recruiting from the community. Potential barriers include characteristics and stigma associated with the condition being studied, differences in cultural expectations and experience with research across study sites, and challenges engaging community healthcare providers and systems. However, because translational trials evaluate the effectiveness of interventions within the community [2], these studies are crucial to evaluating the implementation of these interventions in real-world contexts.

Mental health studies can be particularly challenging for recruitment. For those with major depressive disorder (MDD), withdrawal from regular activities is common. Eligible people may be less likely to engage and more difficult to retain in clinical trials, even if leaving the house to attend study appointments is not required [3]. A review of recruitment to depression treatment trials offering cognitive behavioral therapy (CBT) by computer reported enrollment rates of 2%–60% and uptake rates (actually participating in the treatment) of 3%–25%, representing the wide range of variability in acceptance of and participation in depression treatment trials [4]. The stigma of mental health conditions is an additional barrier for many [5]. Participating in mental health studies requires both acceptance of the condition and willingness to address it. This may create a significant struggle for the participant if shame or stigma are involved [6].

An individual’s culture, social background, and community also powerfully influence their perception of research and likelihood of participation [7, 8]. In many rural, medically underserved areas, patients commonly have a low opinion of the quality of their locally available healthcare and limited direct experience with participation in clinical research [9]. Research teams seen as an extension of this establishment may be viewed with skepticism [10]. In this setting, investigators need to build trust and perform recruitment activities simultaneously. This trust takes time to build and establish and relies on relationships among researchers, patients, and care providers. For example, the Appalachian region is well-known for its overrepresentation of a number of serious medical problems [11]. Despite the need, there are significant misgivings of the medical system among many people in this region [9]. This can lead to delays in care and resistance to enrolling in trials [12, 13].

Multicenter trials necessarily deal with these cultural values at each of the different study sites, with each community having its own sets of values. Efforts to build trust and creative solutions to study-related problems that work in one area may not work in another. This represents an added barrier to successfully recruiting and demands even more effort and flexibility among members of the research team.

The purpose of this paper is to present a case study of recruitment and retention for Program ACTIVE II, a successful, NIDDK-funded behavioral health trial [14]. We present our recruitment strategy, barriers, and costs associated with this trial.

Methods

Study design

Program ACTIVE II was a multisite RCT for adults with type 2 diabetes and MDD [14]. The purpose of the study was to test the comparative effectiveness of manualized (i.e. conducting an intervention according to a manual) exercise (EX) and cognitive behavioral (“talk”) therapies (CBT) for the treatment of depression, individually or in combination (EX+CBT), against usual care (UC). The study protocol was approved by the Institutional Review Boards at each of the individual sites (Indiana University 1105005684, 1308973934; Ohio University 11F031; West Virginia University H-23246).

Potential study participants were identified using a variety of methods, described below, from the communities surrounding the three participating study sites. Research staff trained in the study protocol and approved by the local site Institutional Review Boards (IRB) conducted all contacts with potential participants to assess their interest and eligibility. During screening calls, information was provided about the study using an approved script. If interested, participants gave verbal consent to continue with eligibility screening. Self-reported medical information was collected (see Table 1 for medical exclusion criteria). Those not excluded based on medical information were then screened for psychiatric eligibility. Potential participants who were eligible at medical and psychiatric screening by phone were invited to an in-person baseline eligibility assessment (baseline) where each potential participant provides written informed consent before completing any study-related activities. These baseline assessments were held at fitness centers within the three areas where recruitment took place. Each person then completed a psychiatric interview by phone. Following all baseline activities, an enrollment committee formally evaluated baseline results to determine eligibility and appropriateness for the study for each potential participant. Those not eligible were referred to other services. Eligible participants were randomized, notified of their randomization group (EX, CBT, EX+CBT, or UC), and assigned to an intervention provider, if relevant. Details of the study design and eligibility criteria have been detailed elsewhere [15].

Table 1. Study exclusion criteria

Medical exclusions

Psychiatric exclusions

• Age 18 years or older • Not able to walk • Currently pregnant or planning on becoming pregnant in the next year • History of diabetic ketoacidosis • Continuous insulin therapy since type 2 diabetes diagnosis • History of stroke or transient ischemic attack • Recent cardiac event (diagnosed angina, PTCA, any cardiac intervention for CAD or tachydysrhythmias in the past 6 months) • Laser surgery for proliferative retinopathy in the past 6 months • Aortic stenosis or other sever valvular heart disease or atrial fibrillation • Lower limb amputation • Asensory proliferative retinopathy • Uncontrolled hypertension • Current use of daytime oxygen (severe COPD) • Class III or IV heart failure • Diagnosis of type 2 diabetes < 1 year

• No diagnosis of current MDD • Active suicidal ideation or history of suicide attempt • Bipolar depression • Evidence of psychotic symptoms or history of psychotic disorder • Current substance abuse or dependence • Co-morbid anxiety or eating disorders where MDD is not the primary presentation • If on antidepressant medication, stable dose and usage for at least 6 weeks • Current therapy for MDD by a mental health professional

PTCA Percutaneous transluminal coronary angioplasty, CAD Coronary artery disease, MDD major depressive disorder, COPD chronic obstructive pulmonary disease

Recruitment approach

Participants were recruited from rural southeastern Ohio (OH), north-central West Virginia (WV), and central Indiana (IN) communities. To ensure that recruitment goals were met, the study utilized multiple recruitments approaches, continually assessing the effectiveness of each, and adjusting strategies as necessary [16]. We classified recruitment identification strategies by who made the initial contact. Initial contacts initiated by the potential participant were classified as Inbound and those the study team initiated as Outbound.

Inbound recruitment strategies

Inbound recruitment used a flexible, multifaceted approach, involving outreach to physicians, health departments, libraries, grocery stores, pharmacies, advertisements in newspapers, and flyers posted in public areas. As recruitment methods were implemented, each was reviewed for effectiveness, altering or expanding as necessary to meet recruitment goals.

The main initial recruitment strategy for the study was via physician practices. This included providers affiliated with the study and those seeing patients with diabetes (e.g. endocrinologists, primary care providers, nurse practitioners). Providers were presented study information and asked if contact with patients could occur (e.g. letters to patients, in-office contact, phone calls to patients). Patients who received study information were instructed to call the study team. This outreach was later expanded to include pharmacists, diabetes educators, and diabetes education classes. Study staff also attended health fairs hosted by healthcare providers, community organizations, social justice groups, employer-hosted employee health fairs, and diabetes education groups, providing information about diabetes and depression, study promotional materials, and study flyers. Additional flyers and posters were distributed throughout the communities at libraries, churches, community centers, pharmacies, grocery stores, and to health departments to distribute at diabetes classes.

Advertisements were placed in local community newspapers. Limited advertising was also instituted on Facebook, targeting adult users with an interest in diabetes. Further outreach occurred via radio public service announcements (at the OH site), television segments (OH and WV), and radio segments (IN). These media spots aired locally on public radio and television stations as well as during local evening news broadcasts.

Additional outreach was conducted by direct mail or email to fitness facility members (at the IN and OH sites) and patient lists (WV and IN). Study emails were also sent to registrants of the Indiana Clinical and Translational Sciences Institute’s (CTSI) clinical trials registry (INResearch.org). At the IN site, advertisements were placed on the university’s electronic classifieds board, accessible by students, staff, and faculty.

Outbound recruitment strategies

Electronic medical records from physician practices and volunteer research registries were used to identify prospective participants, who were then directly contacted by the study team. This strategy was utilized at the IN and WV sites. WV conducted outreach to local physician offices. Interested practices allowed the study team contact patients first by mail and then by phone. The IN site partnered with ResNet, a service of the Indiana CTSI, to access patient data in the Eskenazi and IU Health systems. In addition, the IN site utilized the Indiana CTSI’s INresearch.org participant registry. After sending initial emails introducing the study, registrants with type 2 diabetes were contacted by a research assistant. The Ohio University IRB did not approve Outbound calling at the OH site. Unlike the other two sites, OH did not have institutional agreements with healthcare systems to share protected health information with researchers to facilitate Outbound outreach.

Analyses

All contacts with potential participants were logged at the time of contact into a study database. Contact information included the date, time, and nature of the contact. Logged contacts included phone calls (such as study introduction calls, screening, scheduling, psychiatric interviews, etc.), mailings (such as appointment scheduling letters), and in-person appointments (such as eligibility assessments). In addition to logging the contact, the time needed to complete the contact was also recorded, allowing us to track the effort spent on various recruitment tasks. These records were reviewed to determine time spent on recruitment activities, broken down into phone screening and baseline eligibility assessment contacts. Descriptive statistics were used to characterize count, cost, and proportions of individuals in each outcome category. Qualitative data were collected from process notes recorded from team meetings over the study period.

Results

Quantitative data

Individuals screened and enrolled

Figure 1 characterizes recruitment flow using Inbound and Outbound strategies. A total of 18,925 potential participants were attempted contact, representing all patients on call lists for Outbound recruitment (n = 18,067) plus individuals who responded Inbound to community-based recruitment (n = 858) across the three study sites. For Inbound callers, 143 screened eligible for baseline while 464 were ineligible during phone screening. Of the 143 Inbound respondents who were referred for baseline, 74 enrolled in the study.

Fig. 1 [Images not available. See PDF.]

Enrollment flow chart: Inbound vs Outbound

For Outbound recruitment, 154 people were eligible for baseline while 4220 were ineligible at phone screening. Another 6000 individuals declined participation and 7693 could not be reached for screening. Of the 154 respondents who were referred for baseline, 66 completed the assessment and enrolled in the study. The enrollment rate across the population in which contact was attempted was approximately 1% (140 enrolled divided by 18,925 people total).

Differences in recruitment rates were observed between the Inbound and Outbound approaches (Fig. 1 and Table 2). Forty-three percent of individuals could not be contacted from the Outbound calling list while 12% of Inbound respondents could not be re-contacted. Of those who were reached, the rate of self-withdrawal from consideration was three times higher in the Outbound approach compared with Inbound approach (58% vs 20%). For those who were interested and completed phone screening, ineligibility was frequent in both groups though notably higher in those contacted using the Outbound call strategy (96% vs 76%). Although the numbers of Outbound (n = 18,067) and Inbound (n = 858) pools of people vastly differed, similar numbers of individuals screened eligible for baseline (Outbound = 154 versus Inbound =143). Of those who were reached for phone screening, the success rate of ultimately enrolling the study participants was 15 times higher in the Inbound calling group compared with the Outbound calling group (9.7% vs 0.6%).

Table 2. Screening and enrollment numbers and costs

Table 3. Study limitations and potential biases

Study limitations	Potential biases
• Staff effort may be underestimated due to inconsistent logging of time • Reported costs have not been adjusted for inflation	• Stringent study exclusion criteria increased recruitment effort and costs • Recruitment of people with MDD made recruitment more difficult • Some staff members were not consistent in logging time spent on tasks, resulting in some estimation of effort

Effort required to recruit and enroll participants

Recruitment efforts for the study began in May 2012 at the WV and OH sites (see Fig. 2). Based on data collected during the pilot study (R34DK71545) [17], we anticipated that it would be feasible to recruit the entire sample from the WV and OH sites. However, because recruitment rates at these two sites were lower than expected, recruitment began at the IN site in December 2013 to expedite enrollment. December 2013 also marked the start of Outbound calling at the WV site. As a result of these changes, the recruitment rate rose from 5.4 participants per quarter to 14.2 participants per quarter. This rate continued to February 2015 when recruitment ceased at the OH site. Because WV started recruitment without the use of Outbound calling, the enrollment rate there demonstrates the effectiveness of the Outbound strategy by increasing the average quarterly enrollment rate from 0.6 to 2.9 participants enrolled per quarter.

Fig. 2 [Images not available. See PDF.]

Cumulative participant enrollment

Table 4. Key points

• Recruitment to clinical trials is challenging • Recruitment planning during the grant application phase needs to reflect a realistic timeline and adequate budget • Recruitment efforts for Program ACTIVE II cost an average of $1358 for each of the 140 enrolled participants over four years • Sufficient time, funds, and funder support as well as flexibility and adaptation are necessary to meet recruitment goals

Several key developments during the recruitment period are evident in Fig. 2. These included direct mailings by the OH and WV sites in February 2013, the initiation of Outbound recruitment strategies by the IN and WV sites in January 2014, cessation of recruitment efforts at the OH site in February 2015, and an increase in incentives to participants to enhance recruitment and retention in February 2014 and July 2015.

In order to estimate staff effort needed to achieve recruitment goals, average times for each contact activity were calculated. Group mean values were used to estimate for the missing entries. For all phone screening recruitment efforts, the estimated cumulative time spent on these efforts was 3567 h, the majority of which was consumed by Outbound calling (3209 h). An average of 25.5 h of personnel time was spent on all phone recruitment activities for each person enrolled (Table 2).

A total of 1397 h of staff time was spent on baseline activities for the 297 people who screened eligible by phone. This included time for scheduling assessments, mailing letters, in-person assessment activities, and the full psychiatric interview. Of the 297, 140 people completed baseline and were eligible for study participation. The average time for eligibility assessments to reach 140 participants was 10 h per enrolled participant. Adding the baseline time to the phone screening time, a total of 4964 h was required for all direct recruitment and enrollment efforts, an average of 35.5 h per participant enrolled, to obtain the final enrollment of 140 participants.

Costs of participant recruitment

Staff labor and advertising costs were calculated to determine the total cost of recruitment per participant. Staff labor rates were figured at $23.59 per hour to include both salary and benefits. The total calculation of all eligibility screening activities was $117,093 (see Table 2). Outbound calls accounted for the majority of recruitment expenditures ($75,709) due to the time involved. The remaining phone screening figure, $8435, was attributable to Inbound calls. Baseline activities added $32,949 in expenditures. The average amount to screen and enroll each participant into the study was $836.

Additional funds were used for advertising and outreach. Across all three sites, an estimated $72,963 was spent on newspaper advertising, printing posters, and postage for direct mailings. This figure is comparable to the Outbound calling efforts. Total sum of all recruitment, screening, and enrollment activities was estimated at $190,056, which represents an average of $1358 per participant enrolled in the study.

Qualitative observations

The study team across all three sites met 1–2 times per month throughout the funding period. During these meetings, the study investigators discussed a variety of barriers, giving context to the extraordinary effort required by the study teams to meet recruitment goals. These barriers included geographic factors, institutional factors, study personnel factors, and cultural and social barriers, as we describe next.

Geographic factors

The differing geographical and cultural landscape at each of the study sites provided opportunities and challenges for recruitment. Physical distance and transportation infrastructure posed challenges to recruitment efforts at the rural OH and WV sites. The lack of public transportation in rural communities presented a unique problem to recruitment that could have been a factor in motivation to participate in the study. Participants were not compensated for travel to and from assessment sites and some locations for assessments were a fair distance from residences. Ohio and West Virginia participants lived an average of 11 miles and 13.5 miles from the closest assessment sites, respectively, which were located in communities surrounding the study sites in order to be closer to the population areas where the participants lived. However, some participants at these sites lived as far as 20–25 miles from the nearest assessment site, for a round trip as much as 50 miles away from the locations were the study procedure would take place. This present a significant burden to the participants, particularly those who were randomized to receive both CBT and exercise interventions, for as many as two appointments per week during the intervention period.

Institutional factors

The recruitment strategies were influenced by personnel composition and the research climate within each university. Ohio recruitment took place at Ohio University, which is predominantly a teaching institution. While Ohio University has ties to many of the 29 Appalachian counties in southern Ohio, the history and reach of clinical trials have traditionally been on a smaller scale. Formal collaborative partnership agreements with private practices and health systems in the surrounding area did not exist before the start of the study and needed to be established on an individual basis. A total of five healthcare systems partnered with OH over the course of the study. A request to the Ohio University IRB to make Outbound calls to potential participants was denied, limiting this site to the use of Inbound recruitment strategies.

West Virginia University has a long-standing history of clinical research, with the majority being conducted in the basic sciences and T1 translational trials. Institutional infrastructure leveraged for recruitment at this site included partnership with the West Virginia University CTSI, collaborative research agreements between the university and multiple healthcare systems serving northern WV, and a large-scale electronic medical record system from which eligible patients could be identified for recruitment.

In Indiana, recruitment was conducted at the Indiana University School of Medicine (IUSM), the lead study site. IUSM along with the Indiana CTSI have long-standing collaborations with the Eskenazi Health and Indiana University Health systems to collaborate on participant recruitment for clinical trials. These agreements, in partnership with the Regenstrief Institute’s Data Core, facilitated access to patient lists in order to conduct Outbound recruitment by identifying potentially eligible participants. Additional assistance was provided by the Indiana CTSI Research Network (ResNet), whose research assistants performed the majority of Outbound calling to medically screen interested patients and refer eligible patients to the study team.

Institutional differences across the sites expanded or limited the options available to investigators in reaching potential participants. While all sites were influenced by structural and regulatory changes in healthcare markets and delivery that occurred during the study period from 2012 to 2016, each site entered this period of change with different characteristics, resulting in more or less dramatic changes during the study period. For example, the communities around OH underwent a dramatic transformation in the consolidation of many small private primary care practices into larger healthcare organizations. Contractual contingencies on productivity expectations of physicians in these organization placed an implicit and, at times, an explicit limitation on physician efforts to convey study information to patients. While business managers and physicians acknowledged the benefits of how the study could complement their work and the potential benefits for patients, they were hindered by time and resource restrictions, which is consistent with the finding of other researchers in rural environments with limited resources [18].

Study personnel factors

The organization of the study teams contributed to relative strengths and challenges for each site. For example, at WV, study staffing was initially provided by graduate students. While this provided high-quality talent, the nature of the program limited the amount of time students could serve as study employees. This staff turnover contributed to challenges associated with establishing relationships with community organizations. A full-time project coordinator was hired in year 3, which facilitated greater continuity in building relationships with community partners as well as planning and implementation of new strategies. At OH, development of a new research center precipitated changes in study staffing, which ultimately precluded ongoing participation of OH in further enrollment.

Cultural and social barriers

Cultural barriers to recruitment included the burden of living with MDD, limited prior experience with clinical trials, and stigma associated with depression. Initial Inbound recruitment approaches needed potential participants to be sufficiently motivated to call a toll-free number. Since study inclusion criteria required the diagnosis for MDD, it is likely that some potential participants were unable to participate due to the manifest symptoms of depression such as hopelessness, lack of energy, anhedonia, and impaired social interactions. A significant number (296 people, 34.5%) who made the call to the toll-free number did not meet full criteria for MDD, which supports the hypothesis that people with higher burden of depression were less likely to contact the study team. Recruitment strategies such as mailings and fliers posted at clinics, doctor’s offices, churches, and community centers have a lower response rate when compared to recruitment strategies such as the use of a professional interviewer to follow up a mailing or word of mouth [19, 20].

The target communities at each study site differed in levels of experience with clinical trials. Individuals living in rural areas have limited exposure to clinical trials compared to their urban counterparts, which appeared to contribute to greater hesitancy to respond to Inbound recruitment efforts. Perceptions of researchers as outsiders and the perception of participation in research as being a “guinea pig” contributed to hesitancy to engage in screening.

The benefits of participating in a RCT may have been viewed with reluctance, especially given that participants could not be promised tangible benefits other than what they may receive by being randomly assigned to a treatment group. As a result, randomization to a specific treatment group without choice may have dissuaded individuals from volunteering. People who are seeking treatment for MDD frequently do not wish to be assigned to a Usual Care group, risking the possibility of receiving no treatment for the three months of the intervention period. This resulted in some eligible participants declining to participate in the study. Additionally, the possibility of being assigned to a counseling group may have influenced participation due to stigma associated with counseling.

At the IN site, clinical trials are more familiar to potential participants, but those trials typically involve fewer demands on participant time and engagement. Participants employed in hourly positions working in service industries consistently reported difficulty in knowing their work schedules far enough in advance to be able to keep study appointments. As a result, adaptations were made to accommodate work schedules. In addition, the target participant population at this site had an expectation of payment for study participation. A decision to institute and increase participant incentives across all sites was made on the basis of these public expectations.

Discussion

Community recruitment of participants is critical to the advancement of all forms of science, particularly behavioral translational trials. Increasingly, clinical trials must compete for participant attention, interest, and time against a backdrop of considerable life demands and opportunities to easily obtain reliable health information. Funders and investigators recognize the central role that recruitment plays in the success of funded research. Underestimation of the effort and costs needed for successful recruitment is a modifiable variable that investigators can and should address during the study planning phase of the research. Here, the Program ACTIVE II study [14] experience is shared as a case example of the breadth and depth of recruitment strategies that were required to overcome barriers to study success as well as the monetary costs incurred (Table 3).

Endemic to the Program ACTIVE II study design, we encountered the following challenges: the specific eligibility criteria of the study (duration of type 2 diabetes for at least one year while meeting full DSM-IV-TR criteria for current MDD; medically appropriate for community-based exercise) [21], requirements for study participation (attending 2–3 appointments required for the baseline assessment, 10 weekly appointments for the CBT intervention, six exercise classes over eight weeks for the exercise intervention, and two appointments for each of three follow-up assessments) [21–23], stigma associated with both type 2 diabetes and depression [24], participant beliefs about study participation (e.g. lack of personal gain) [23, 24], diversification of communication channels to reach potential participants [22], health literacy issues [22], and co-morbid illnesses that may impede participation in studies [22, 23]. Each of these characteristics may have been relevant during study recruitment and reflected in the effort necessary for successful study enrollment.

These findings from the Program ACTIVE II recruitment experience demonstrate the considerable time, effort, and costs associated with recruiting participants for this federally funded behavioral RCT. These expenditures far exceeded investigator expectations and demonstrated important lessons for investigators and sponsors of future clinical trials. The costs incurred specifically for recruitment activities were nearly $1400 per participant, with $836 of this amount attributable to the direct costs required for recruitment of each enrolled participant, in line with direct recruitment costs reported by other diabetes studies [25, 26]. While grant application budgets tend to be focused on the costs associated with interventions, recruitment costs are substantial and should not be underestimated.

A second lesson was the amount of time needed to meet our recruitment target. Our team anticipated that we would complete recruitment activities within the first 24 months of the funding period. Instead, we required 4.5 years to achieve our goal. The extension of this period was a function of the large number of people we needed to contact to compete enrollment. The recruitment rates of this study (0.6%–1%) were significantly lower than those observed in our previous pilot study (8%) that used identical eligibility criteria [17], though similar to the 1.3% recruitment rate experienced by another depression treatment trial [27]. In the Program ACTIVE pilot work, Inbound recruitment approaches were successfully used to achieve a smaller target sample of 50 [17]. However, replication of those same methods did not yield similar recruitment rates in the larger trial at the same site (Table 4).

Continual reassessment of strategies and adaptations to the recruitment plan were needed throughout the recruitment period. The Outbound calling method was particularly important for Program ACTIVE II as people with depression may not seek mental health treatment because of stigma [5, 28, 29] or take the initiative to make the initial contact with a study [30]. This was especially relevant for our study considering that we recruited people with current MDD, who are relatively unlikely to initiate treatment for their depression [31].

There are limitations to the data presented in this paper. First, tracking of staff activities was not the primary goal of study data collection so some estimation was required in our calculations. As a result, the total effort of study staff may be underestimated for some activities (we erred on the side of underestimation). Additionally, figures reported are based on actual costs over the period of recruitment (2012–2016) and have not been adjusted for inflation. These numbers should be adjusted by investigators for use in future research budgets.

A critical element in the ultimate success of this trial was the flexibility and continued support by the funding agency as the investigators adapted to barriers to recruitment. The agency permitted additional time to complete recruitment and demonstrated confidence in the investigators, who were the most knowledgeable about their local barriers and needs. Had restrictions been placed on study funding based original assumptions about recruitment rates and strategies, the investigators would not have been able to demonstrate the effectiveness of the interventions [32].

Conclusions

Recruitment remains the primary challenge to clinical trials. Innovation and flexibility in response to changes in the environment are critical to success. It is imperative that investigators plan conservatively for recruitment effort and costs during the grant application stage. Despite significant recruitment challenges, investigators should be mindful of meeting recruitment milestones or risk continued funding. The percentage of studies being funded has decreased substantially over the past 10 years [32], so it is imperative that those trials that are funded are successful in reaching their stated recruitment targets. However, not all conditions can be anticipated so investigators must be allowed flexibility as well. Policies that place study funding contingencies based upon achieving short-term recruitment targets should balance the risk of expenditures of failed studies against the loss of scientific knowledge and intervention development due to premature termination of studies requiring additional time and resources to meet recruitment goals.

Acknowledgements

The Program ACTIVE Research Team wishes to thank the following individuals for their many contributions to this study: Rachel Clift, RN; Susan Eason, M.A.; David Donley, M.S.; Alex Tylka, M.S.; Lindsey Sams, B.S.; Jaclyn Babe, B.S.; Christiaan Abildso, Ph.D; Daniel Bonner, M.S., Melinda Ruberg, B.S.; Frank Schwartz, M.D., Bernadette Heckman, Ph.D.; Cammie Starner; Lynn Petrik; Tracey Garrett; Molly Long, M.S.; Michael Craven, M.S.; Allysia Ramirez, M.S.; Amber Massa, M.S.; Chelsea Holbert, M.A.; Debby Wimer; Kelly Chaudoin, M.A.; Sarah Mielens, M.A.; Ellen Knapp, M.A.; Kent Crick, M.S.; Danielle Epler, M.S.; Michelle Weinstein, M.S.; Kasey Goodpaster, Ph.D.; Brett McKinney; and Kisha Wilkerson.

The authors also thank all of the participants of Program ACTIVE II who gave generously of their time and energy to engage in a new research activity. We are grateful for their generosity of spirit and their willingness to try something new.

Authors’ contributions

BAM analyzed and interpreted the data. All authors contributed to the study design, writing, reviewing, and editing of the manuscript and approve the final manuscript.

Funding

This work was supported by the National Institute of Diabetes and Digestive and Kidney Diseases (grant number R18DK092765). The funder provided no involvement in the preparation of this manuscript nor in the design of the study, collection of the data, analysis or interpretation of the data, the writing of any manuscripts related to this study, or in the decision to submit any articles for publication.

Availability of data and materials

The datasets used and/or analyzed during the current study area available from the corresponding author on reasonable request.

Ethics approval and consent to participate

This study received Institutional Review Board approval at each of the three study sites (Indiana University 1105005684, 1308973934; Ohio University 11F031; West Virginia University H-23246). All participants provided written consent prior to participating in the study and verbal consent prior to the initial phone screening. Consent for publications. This manuscript does not contain any individual person’s data in any form.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Abbreviations

CBT

Cognitive Behavioral Therapy, or “talk therapy,” intervention

EX

Exercise intervention

EX+CBT

Combined exercise and talk therapy intervention

IN

Indiana University Indianapolis study site

MDD

Major depressive disorder

OH

Ohio University study site

UC

Usual Care control

WV

West Virginia University study site

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Cohen, J. The statistical power of abnormal-social psychological research: a review. J Abnorm Soc Psychol; 1962; 65, pp. 145-153.[COI: 1:STN:280:DyaF38%2FitFGksg%3D%3D] [DOI: https://dx.doi.org/10.1037/h0045186]

2. Ford, I; Norrie, J. Pragmatic Trials. N Engl J Med; 2016; 375, 5 pp. 454-463. [DOI: https://dx.doi.org/10.1056/NEJMra1510059]

3. Woodford, J; Farrand, P; Bessant, M; Williams, C. Recruitment into a guided internet based CBT (iCBT) intervention for depression: lesson learnt from the failure of a prevalence recruitment strategy. Contemp Clin Trials; 2011; 32, 5 pp. 641-648. [DOI: https://dx.doi.org/10.1016/j.cct.2011.04.013]

4. Kaltenthaler, E; Sutcliffe, P; Parry, G; Beverley, C; Rees, A; Ferriter, M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med; 2008; 38, 11 pp. 1521-1530.[COI: 1:STN:280:DC%2BD1cnkvVejug%3D%3D] [DOI: https://dx.doi.org/10.1017/S0033291707002607]

5. Thorngren JM. Rural mental health: a qualitative inquiry. J Rural Community Psychol. 2003;E6(2):1–11.

6. Fisher, L; Hessler, D; Naranjo, D; Polonsky, W. AASAP: a program to increase recruitment and retention in clinical trials. Patient Educ Couns; 2012; 86, 3 pp. 372-377. [DOI: https://dx.doi.org/10.1016/j.pec.2011.07.002]

7. Ford, JG; Howerton, MW; Lai, GY; Gary, TL; Bolen, S; Gibbons, MC et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer; 2008; 112, 2 pp. 228-242. [DOI: https://dx.doi.org/10.1002/cncr.23157]

8. Gorelick, PB; Harris, Y; Burnett, B; Bonecutter, FJ. The recruitment triangle: reasons why African Americans enroll, refuse to enroll, or voluntarily withdraw from a clinical trial. An interim report from the African-American Antiplatelet Stroke Prevention Study (AAASPS). J Natl Med Assoc; 1998; 90, 3 pp. 141-145.[COI: 1:STN:280:DyaK1c3htFWntQ%3D%3D] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/9549977][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2608331]

9. Nesbitt, TS; Marcin, JP; Daschbach, MM; Cole, SL. Perceptions of local health care quality in 7 rural communities with telemedicine. J Rural Health; 2005; 21, 1 pp. 79-85. [DOI: https://dx.doi.org/10.1111/j.1748-0361.2005.tb00066.x]

10. Paskett, ED; Cooper, MR; Stark, N; Ricketts, TC; Tropman, S; Hatzell, T et al. Clinical trial enrollment of rural patients with cancer. Cancer Pract; 2002; 10, 1 pp. 28-35. [DOI: https://dx.doi.org/10.1046/j.1523-5394.2002.101006.x]

11. PDA I, The Cecil B. Sheps Center for Health Services Research. Health care costs and access disparities in Appalachia; 2012; Washington, Appalachian Regional Commission:

12. Hansen, MM; Resick, LK. Health beliefs, health care, and rural Appalachian subcultures from an ethnographic perspective. Fam Community Health; 1990; 13, 1 pp. 1-10. [DOI: https://dx.doi.org/10.1097/00003727-199005000-00003]

13. Small, C. Giger, JN; Davidhizar, RE. Appalachians. Transcultural nursing: assessment and intervention; 1991; St. Louis, Mosby, Inc.:

14. de Groot, M; Shubrook, JH; Hornsby, WG, Jr; Pillay, Y; Mather, KJ; Fitzpatrick, K et al. Program ACTIVE II: outcomes from a randomized, multistate community-based depression treatment for rural and urban adults with type 2 diabetes. Diabetes Care; 2019; 42, 7 pp. 1185-1193. [DOI: https://dx.doi.org/10.2337/dc18-2400]

15. de Groot M, Shubrook J, Schwartz F, Hornsby WGJ, Pillay Y. Program ACTIVE II: design and methods for a multi-center community-based depression treatment for rural and urban adults with type 2 diabetes. Diabetes Res Ther. 2015;1(1):1–11.

16. Nicholson, LM; Schwirian, PM; Groner, JA. Recruitment and retention strategies in clinical studies with low-income and minority populations: progress from 2004-2014. Contemp Clin Trials; 2015; 45, Pt A pp. 34-40. [DOI: https://dx.doi.org/10.1016/j.cct.2015.07.008]

17. de Groot, M; Kushnick, M; Doyle, T; Merrill, J; McGlynn, M; Shubrook, J et al. A model of community-based behavioral intervention for depression in diabetes: program ACTIVE. Diabetes Spectr; 2010; 23, 1 pp. 18-25. [DOI: https://dx.doi.org/10.2337/diaspect.23.1.18]

18. Miyamoto, S; Henderson, S; Young, H; Ward, D; Santillan, V. Recruiting rural participants for a telehealth intervention on diabetes self-management. J Rural Health; 2013; 29, 1 pp. 69-77. [DOI: https://dx.doi.org/10.1111/j.1748-0361.2012.00443.x]

19. Ives, DG; Kuller, LH; Schulz, R; Traven, ND; Lave, JR. Comparison of recruitment strategies and associated disease prevalence for health promotion in rural elderly. Prev Med; 1992; 21, 5 pp. 582-591.[COI: 1:STN:280:DyaK3s%2Fms1Sluw%3D%3D] [DOI: https://dx.doi.org/10.1016/0091-7435(92)90066-Q]

20. Rodriguez, MD; Rodriguez, J; Davis, M. Recruitment of first-generation Latinos in a rural community: the essential nature of personal contact. Fam Process; 2006; 45, 1 pp. 87-100. [DOI: https://dx.doi.org/10.1111/j.1545-5300.2006.00082.x]

21. Huang, GD; Bull, J; Johnston McKee, K; Mahon, E; Harper, B; Roberts, JN et al. Clinical trials recruitment planning: a proposed framework from the clinical trials transformation initiative. Contemp Clin Trials; 2018; 66, pp. 74-79. [DOI: https://dx.doi.org/10.1016/j.cct.2018.01.003]

22. Mody, L; Miller, DK; McGloin, JM; Freeman, M; Marcantonio, ER; Magaziner, J et al. Recruitment and retention of older adults in aging research. J Am Geriatr Soc; 2008; 56, 12 pp. 2340-2348. [DOI: https://dx.doi.org/10.1111/j.1532-5415.2008.02015.x]

23. Provencher, V; Mortenson, WB; Tanguay-Garneau, L; Belanger, K; Dagenais, M. Challenges and strategies pertaining to recruitment and retention of frail elderly in research studies: a systematic review. Arch Gerontol Geriatr; 2014; 59, 1 pp. 18-24. [DOI: https://dx.doi.org/10.1016/j.archger.2014.03.006]

24. Levkoff, S; Sanchez, H. Lessons learned about minority recruitment and retention from the Centers on Minority Aging and Health Promotion. Gerontologist; 2003; 43, 1 pp. 18-26. [DOI: https://dx.doi.org/10.1093/geront/43.1.18]

25. Effoe, VS; Katula, JA; Kirk, JK; Pedley, CF; Bollhalter, LY; Brown, WM et al. The use of electronic medical records for recruitment in clinical trials: findings from the Lifestyle Intervention for Treatment of Diabetes trial. Trials; 2016; 17, 1 496. [DOI: https://dx.doi.org/10.1186/s13063-016-1631-7]

26. Blackwell, CS; Foster, KA; Isom, S; Katula, JA; Vitolins, MZ; Rosenberger, EL et al. Healthy living partnerships to prevent diabetes: recruitment and baseline characteristics. Contemp Clin Trials; 2011; 32, 1 pp. 40-49. [DOI: https://dx.doi.org/10.1016/j.cct.2010.10.006]

27. LaRocco-Cockburn, A; Reed, SD; Melville, J; Croicu, C; Russo, JE; Inspektor, M et al. Improving depression treatment for women: integrating a collaborative care depression intervention into OB-GYN care. Contemp Clin Trials.; 2013; 36, 2 pp. 362-370. [DOI: https://dx.doi.org/10.1016/j.cct.2013.08.001]

28. Conner, KO; Copeland, VC; Grote, NK; Koeske, G; Rosen, D; Reynolds, CF, 3rd et al. Mental health treatment seeking among older adults with depression: the impact of stigma and race. Am J Geriatr Psychiatry; 2010; 18, 6 pp. 531-543. [DOI: https://dx.doi.org/10.1097/JGP.0b013e3181cc0366]

29. Barney, LJ; Griffiths, KM; Christensen, H; Jorm, AF. Exploring the nature of stigmatising beliefs about depression and help-seeking: implications for reducing stigma. BMC Public Health; 2009; 9, 61. [DOI: https://dx.doi.org/10.1186/1471-2458-9-61]

30. Bell, KR; Hammond, F; Hart, T; Bickett, AK; Temkin, NR; Dikmen, S. Participant recruitment and retention in rehabilitation research. Am J Phys Med Rehabil; 2008; 87, 4 pp. 330-338. [DOI: https://dx.doi.org/10.1097/PHM.0b013e318168d092]

31. Blumenthal, R; Endicott, J. Barriers to seeking treatment for major depression. Depress Anxiety; 1996; 4, 6 pp. 273-278. [DOI: https://dx.doi.org/10.1002/(SICI)1520-6394(1996)4:6<273::AID-DA3>3.0.CO;2-D]

32. National Institutes of Health. Research Project Success Rates by NIH Institute for 2017; 2018; https://report.nih.gov/success_rates/success_byic.cfm