eRegCom—Quality Improvement Dashboard for healthcare providers and Targeted Client Communication to pregnant women using data from an electronic health registry to improve attendance and quality of antenatal care: study protocol for a multi-arm cluster randomized trial

Abstract

Background

This trial evaluates interventions that utilize data entered at point-of-care in the Palestinian maternal and child eRegistry to generate Quality Improvement Dashboards (QID) for healthcare providers and Targeted Client Communication (TCC) via short message service (SMS) to clients. The aim is to assess the effectiveness of the automated communication strategies from the eRegistry on improving attendance and quality of care for pregnant women.

Methods

This four-arm cluster randomized controlled trial will be conducted in the West Bank and the Gaza Strip, Palestine, and includes 138 clusters (primary healthcare clinics) enrolling from 45 to 3000 pregnancies per year. The intervention tools are the QID and the TCC via SMS, automated from the eRegistry built on the District Health Information Software 2 (DHIS2) Tracker. The primary outcomes are appropriate screening and management of anemia, hypertension, and diabetes during pregnancy and timely attendance to antenatal care. Primary analysis, at the individual level taking the design effect of the clustering into account, will be done as intention-to-treat.

Discussion

This trial, embedded in the implementation of the eRegistry in Palestine, will inform the use of digital health interventions as a health systems strengthening approach.

Trial registration

ISRCTN Registry, ISRCTN10520687. Registered on 18 October 2018

Background

Scale up of effective, high-quality interventions is essential in order to reach the Sustainable Development Goals and achieve Universal Health Coverage in maternal and child healthcare [1, 2]. Antenatal care (ANC), postpartum care (PPC), and newborn care in low- and middle-income countries show low effective coverage [3, 4].

Countries can improve healthcare systems by exploiting the potential of digital technology. Digital health information systems with individual-level data and mobile technologies are expanding globally and provide an opportunity to support, involve, and influence healthcare providers and their clients’ behavior [5–7]. However, effective digital health interventions are complex and require careful design, implementation, and evaluation [8].

Audit and feedback, widely used quality improvement interventions, allow healthcare providers to assess and adjust their performance and, as a result, improve the quality of the care they deliver [9, 10]. Audit and feedback can be defined as any summary of clinical performance of healthcare over a specified period of time and may include recommendations for clinical actions [11]. It is most effective when it is presented more than once, given both verbally and in writing, provided by a supervisor or respected colleague, presented with clear goals and action plans, aimed to decrease targeted behavior, directed towards the recipient with room for improvement, and to health professionals typically working with guideline-bound clinical activities [12]. The model of actionable feedback emphasizes that feedback should be timely, individualized, non-punitive, and meaningful to the recipient to be effective [13]. Explicit use of theory to inform intervention development, and user involvement in all stages of a digital health intervention development, implementation, and evaluation, is recommended, but seldom described [14–17].

Digital Targeted Client Communication (TCC) can utilize data, such as demographic characteristics and health status, to tailor communications to an individual’s specific needs if good-quality individual-level data are available [18, 19]. Short message service (text message (SMS)) is often used as a medium to deliver TCC interventions. Appointment reminders alone, or coupled with generic health promotion messages, can impact people’s knowledge, health literacy, and attitudes and thus improve healthy behavior and utilization of healthcare services [17, 20–23]. However, even though tailored, co-designed, theory-driven TCC interventions tend to be more effective than generic messages, evidence is needed, especially in the field of maternal and child health (MCH) in low- and middle-income countries [15, 19, 21, 22].

Palestinian context

The total population in Palestine is 4.17 million, 2.58 million (62%) in the West Bank and 1.59 million (38%) in the Gaza Strip [24]. There are approximately 62,000 live births in the West Bank and 56,000 live births in the Gaza Strip annually. Individuals under 18 years of age constitute 48% of the total population, and the fertility rate is 3.8 in the West Bank and 4.9 in the Gaza Strip.

MCH is an important area in the Palestinian healthcare system, and gestational anemia, hypertensive disorders of pregnancy, gestational diabetes mellitus, and fetal growth restriction are identified as priority conditions for prevention [25]. Approximately 43% of pregnant women attend public ANC services, but only 13% according to the recommended national schedule, which results in low effective coverage [26]. Both the utilization and quality of health services have room for improvement. MCH supervisors, each responsible for 16 to 54 public primary healthcare clinics (PHCs), supervise midwives, nurses, and community health workers, mainly by checking registers and clinical equipment [27].

The routine governmental documentation tool for ANC and PPC in Palestine is the eRegistry built on the District Health Information Software 2 (DHIS2) Tracker [28]. The eRegistry is a web-based open-source information system for the longitudinal collection, storage, retrieval, analysis, and dissemination of information on health determinants and outcomes for individual persons [5, 29]. The implementation of the Palestinian eRegistry was carried out by the Ministry of Health (MoH) and the Palestinian National Institute of Public Health (PNIPH), with support from the Norwegian Institute of Public Health and the University of Oslo, as a measure to support healthcare providers in screening and management, and improve monitoring and evaluation [30, 31].

The eRegistry in Palestine provides clinical decision support based on national guidelines, including referral recommendations. It creates longitudinal pregnancy records and removes the need for secondary data reporting or manual aggregation for reporting. The eRegistry’s comprehensive information gathered in a continuous fashion allows TCC via SMS to clients and QID for healthcare providers, but the effectiveness is unknown.

Objectives

The primary objectives of the eRegCom cluster randomized controlled trial (CRCT) are, compared to the basic eRegistry functionalities in public MCH PHCs in Palestine, to estimate the effectiveness of the eRegistry’s:

• Quality Improvement Dashboard (QID) on improving appropriate screening and management for anemia, hypertension, and diabetes during pregnancy by the healthcare provider

• Targeted Client Communication (TCC) on improving timely attendance to ANC by the woman

• QiD and TCC interventions combined on the measures described above

We will also estimate the effectiveness across equity measures and, in a sub-sample, assess the effect of the TCC intervention on pregnant women’s worries and perceptions of ANC, including the quality of care and information received from the health system (Fig. 1).

Fig. 1. [Images not available. See PDF.]

The effect of the TCC intervention on pregnant women’s worries in a sub-sample

Methods

Trial design

This CRCT is a superiority trial with four parallel arms (TCC, QID, TCC and QID, and control). The unit of randomization is an individual PHC, with the exception of two clusters. These two clusters include two PHCs each, where the pair of PHCs is served by the same healthcare provider.

Study setting

The most common organizational structure in the public maternal and child healthcare system in Palestine includes two healthcare providers per clinic, mainly midwives, nurses, and/or community health workers, in addition to a doctor serving several clinics. Each user in the governmental eRegistry has a unique username and password, which enable access to records and specific system features according to their assigned role. The woman’s personal ID number is used to create an electronic ANC record for her pregnancy. The sociodemographic, obstetric, and medical information including clinical tests, laboratory, and ultrasound measurement results are entered into the eRegistry at point-of-care. The data trigger different digital health interventions such as the guideline-based clinical decision support and automated public health reports. Women identified with certain risk factors are referred for additional management, and their records are available in the so-called high-risk clinics after referral [31].

Eligibility criteria

All public PHCs offering ANC and PPC services using the eRegistry were eligible for the study. PHCs that enrolled less than 45 or more than 3000 new pregnancies in 2016 were excluded. No exclusion criteria were made based on individual healthcare provider’s or pregnant women’s characteristics (Fig. 2).

Fig. 2 [Images not available. See PDF.]

Flow diagram showing eligible primary healthcare clinics (PHCs) and allocation

A total of 138 clusters (individual PHC and a combination of two PHCs served by the same healthcare provider for two clusters) were eligible for randomization. They enrolled a mean of 344 and a median of 131 new pregnant women in 2016 (Additional file 1). Data from all ANC visits occurring in the included PHCs during the study period will be used to assess our outcomes.

Intervention

The interventions are at the cluster-level targeting nurses, midwives, and community health workers, referred to as healthcare providers, and their clients.

The Quality Improvement Dashboard intervention

The QID intervention is the provision of access to the QID within their routine eRegistry, as well as training of healthcare providers on how to use it.

As part of the QID intervention development process, public healthcare providers, supervisors, and health system administrators were interviewed to map the existing supervision and feedback system [27]. We found that healthcare providers received irregular supervision visits with limited focus on performance improvement at the clinical level. Findings and recommendations regarding benchmarking [32], the use of SMART (Specific, Measurable, Achievable, Relevant and Timely) criteria, concepts from social nudging and Enhanced active choice [33], and the Model of Actionable Feedback [13] informed the design of the QID. eRegistry users’, nursing and medical directors’, and PNIPH and MoH staff’s reviews informed the revisions. The final version was translated into the local Arabic language.

The training curriculum is founded on quality improvement theories and models such as the Plan, Do, Study, Act (PDSA) cycles [34]. Healthcare providers learn how they can use the QID, which presents indicators of quality gaps in their clinic, as a tool to improve the quality of the care they provide.

The QID includes four tabs for each focus area, namely anemia, hypertension, diabetes, and attendance. Healthcare providers are given a new focus area every week through a message in the eRegistry that congratulates a good performing PHC or presents an evidence-based statement on the week’s focus area. Each tab contains performance indicators calculated from data entered at point-of-care. The indicators are presented as an average over the last 3 months, in both tables and graphs, and benchmarked with clinics within the district. The clinic’s performance level—defined as an index of the absolute and relative values of an overall screening and an overall management indicator—drives the appropriate action items for each focus area. Two screening and two management action items that include recommendations for improvement are presented in colors (green = good performance, yellow = room for improvement, red = large room for improvement) with monthly updates (Fig. 3). The healthcare providers can score the action items via a thumbs up/down icon and add written comments.

Fig. 3 [Images not available. See PDF.]

Quality Improvement Dashboard for anemia to healthcare providers from the eRegistry. Top left: individualized on clinic level; top right: reminders of the week’s focus area, namely anemia, hypertension, diabetes, or attendance; tables: indicators with benchmarks; graphs: percentage average over the last 3 months; right: action items presented in colors (green = good performance, yellow = room for improvement, red = large room for improvement)

The Targeted Client Communication intervention

The TCC intervention is automated text messages sent via SMS from the eRegistry to pregnant women, and training of healthcare providers on how to enroll women in the program.

The TCC intervention development process, including the full-text message library, is described in detail elsewhere [35]. In short, pregnant women and healthcare providers were interviewed to identify how pregnant women perceive their risks of getting anemia, hypertension, and diabetes during pregnancy and susceptibility to fetal growth restriction, in addition to benefits and barriers of attending ANC, using the Health Belief Model. The findings, concepts from social nudging and Enhanced active choice [33], and the Model of Actionable Feedback [13] informed the content, medium, timing, and frequency of the TCC intervention.

The training includes how to register and withdraw women, and the timing and the content of the text messages. Healthcare providers register women that agree to receive text messages by ticking a box in the eRegistry. This can be done at any time in gestation, preferably at the first visit.

The tailored text messages include the woman’s name and her clinic’s name and may also include the date of her next appointment (Table 1). Those that include information about one or more of the high priority areas (anemia, hypertension, and diabetes during pregnancy, and fetal growth restriction) will be sent at the time these conditions are screened for, namely at the 16, 18–22, 24–28, 32, or 36 weeks’ gestation routine ANC visit. However, women with documented anemia, hypertension, diabetes, or fetal growth restriction will not receive the information about routine screening for that condition, e.g., women with diabetes will not receive a text message about the routine screening for diabetes at 24–28 weeks’ gestation. The text messages will be sent to the mobile number registered on the woman after working hours during ANC at the following time points:

• At registration, a welcome message including information about how to un-enroll

• One week prior to a timely scheduled routine visit

• Three days prior to a timely scheduled routine visit to women with risk factors for anemia, hypertension, diabetes, or fetal growth restriction

• Twenty-four hours prior to a visit

• Twenty-four hours after a missed timely scheduled routine visit

• Recapture message 24 h prior to the start of the appropriate time window for a routine visit to women without any timely scheduled routine visit in the future

Table 1. Example text messages send to an overweight pregnant woman without identified anemia, diabetes, or hypertension

Welcome	One week prior to a visit	Three days prior to a visit	Twenty-four hours prior to a visit	Twenty-four hours after a missed visit	Twenty-four hours prior to a missing visit
Dear Abi, Most women attend antenatal care for their own and baby’s health. The healthcare provider will measure your blood pressure, hemoglobin and blood glucose level. You will receive text message appointment reminders. Please let us know if you do not want these messages. Tamoon clinic	Dear Abi, The date of your upcoming appointment is 2020.04.20. One in 20 develop high blood pressure in pregnancy, and this may affect your health and the growth of your baby. The healthcare provider will measure your blood pressure and the amount of protein in urine as they may be a sign of high blood pressure. Tamoon clinic	Dear Abi, 2020.04.20 is your next appointment, as agreed. High body weight before pregnancy, may increase the risk of developing high blood pressure. The healthcare provider will measure your blood pressure and the amount of protein in urine at your next visit. Tamoon clinic	Dear Abi, This is a reminder that you have an appointment tomorrow, 2020.04.20, as agreed. Tamoon clinic	Dear Abi, Sorry to have missed you at yesterday’s appointment. We hope to see you back with us for antenatal care as soon as possible. Tamoon clinic	Dear Abi, You haven’t scheduled your next appointment, which is coming up soon. Timely attendance to every antenatal care visit may ensure you and your baby's safety, through early detection and treatment of complications. Please contact us to schedule your next visit. We are ready to provide you with care, and hope to see you soon. Tamoon clinic

Adherence, training, and concomitant care

The TCC and QID interventions in the eRegistry are available to healthcare providers working in PHCs randomized to trial arms receiving one or both of the digital health interventions. The research team trained the trainers, who were eRegistry staff at PNIPH and nursing directors, each representing a governorate. Healthcare providers from TCC and QID intervention PHCs were trained separately, and healthcare providers working in PHCs randomized to both interventions received both trainings. The trainings were conducted in the first quarter of 2019, and each session lasted for half a day. A follow-up training session was conducted in September 2019. Healthcare providers in QID intervention PHCs received a video presenting new functionalities in the QID at the start of the trial. New employees in intervention PHCs will receive on-site training from one of the trainers.

MCH supervisors will carry out similar periodic supervision visits to all PHCs in all arms of this CRCT. Women will not receive any concomitant care across intervention and control PHCs.

Outcomes

The outcomes represent key areas of quality concerns in Palestine and are in line with our previous eRegQual CRCT, a study assessing the effect of using an eRegistry versus paper-based ANC records [25]. The Palestinian ANC guidelines for the recommended gestational week (Table 2) for screening and management of anemia, hypertension, and diabetes have defined our outcomes. For routine visits recommended for one specific week, we have made the time window for the outcome measures 2 weeks wider than the guideline, to allow some flexibility for maternal choice and time for e.g. laboratory results to be received and documented in the eRegistry.

Table 2. Time windows for the outcome measures and routine ANC visits including the primary screening test

	First visit	Routine visits
Recommended occurrence of ANC visits, GW	Any week	16	18–22	24–28	32	36
Included in outcome measures, GW	Any week ± 7 days	15–17	18–22	24–28	31–33	35–37
Primary screening tests for priority conditionsa	Blood pressure, hemoglobin, urine glucose	Blood pressure, fetal growth	Blood pressure	Blood pressure, hemoglobin, blood glucose	Blood pressure, fetal growth	Blood pressure, hemoglobin, fetal growth

ANC antenatal care, GW weeks of gestation

^aAnemia, hypertension, diabetes, and fetal growth restriction according to the Palestinian guidelines

Primary outcomes

Comparison I: QID arm vs control arm

The primary outcome in comparison 1 is the proportion of women who receive appropriate screening and management of anemia, hypertension, and diabetes (Tables 3, 4, 5, 6, and 7). For example for anemia, the proportion is calculated by identifying the number of ANC visits where anemia should have been screened for and/or managed (denominator), and among them where anemia was appropriately screened for and/or managed (numerator). Only the first step in the management algorithm will be included (Tables 3, 4, 5, 6, and 7). Pregnant women with documented ongoing anemia, hypertension, or diabetes prior to the timely routine visit will not be included in the denominator for appropriate screening and management of anemia, hypertension, and diabetes respectively.

Table 3. Anemia screening test, result, and appropriate management

Hemoglobin test at first visit, 24–28, and 36 weeks’ gestation
Hb ≥ 11 g/dL	Hb 10.9–7 g/dL	Hb < 7g/dL
No further action	Hb after 1 month	Refer to hospital

Table 4. Hypertension screening test, result, and appropriate management prior to 20 weeks’ gestation

Blood pressure prior to 20 weeks’ gestation, test at the first visit and at 16, 18–22 weeks’ gestation
SBP < 140 mmHg and DBP < 90 mmHg	SBP ≥ 140 mmHg, DBP ≥ 90 mmHg
No further action	Refer to high-risk clinic

SBP systolic blood pressure, DBP diastolic blood pressure

Table 5. Hypertension screening test, result, and appropriate management at or after 20 weeks’ gestation

Blood pressure at or after 20 weeks’ gestation, test at 18–22, 24–28, and 36 weeks’ gestation
SBP < 140 mmHg and DBP < 90 mmHg	DBP 90–99 mmHg, SBP 140–149 mmHg	DBP ≥ 100 mmHg, SBP ≥ 150 mmHg
No further action	New BP within 4 days	Refer to hospital

SBP systolic blood pressure, DBP diastolic blood pressure

Table 6. Diabetes screening test, result, and appropriate management prior to 24 weeks’ gestation

Urine glucose test at booking prior to 24 weeks’ gestation
Negative	Positive
No further action	Blood glucose test

Table 7. Diabetes screening test, result, and appropriate management at or after 24 weeks’ gestation

Blood glucose test at 24–28 weeks’ gestation
RBG < 105 mg/dL, FBG < 95 mg/dL	RBG 105–140 mg/dL, FBG 95–126 mg/dL	RBG ≥ 140 mg/dL, FBG ≥ 126 mg/dL
No further action	New blood test within 3 weeks	Refer to a diabetes clinic

RBG random blood glucose, FBG fasting blood glucose

Comparison II: TCC arm vs control arm

The primary outcome in comparison 2 is the proportion of all timely routine ANC visits that a woman was eligible for, where the woman attended (Table 2). The first ANC booking visit is excluded, and each timely routine visit will be counted separately as a singular opportunity to succeed or fail in attendance.

Comparison III: QID + TCC arm vs control arm

The primary outcome in comparison 3 is the product of the quality of care (comparison 1) and the utilization of care (comparison 2) to assess the effective coverage. For example for anemia, it is the proportion of all timely routine ANC visits where anemia should have been screened for and/or managed, where the woman attended and anemia was appropriately screened for and/or managed.

Effect on healthcare equity

We will assess the effects of the interventions on the primary outcomes across equity measures used in routine statistics in Palestine. The data points include average monthly household incomes (less than 200; 200–900; 901–1824; 1825–3054; and > 3055 Israeli new Shekel), mother’s years of education (< 10; 10–13 years; > 13 years), age at marriage (less than 20; 21–25; 26–30; 31–35; 36–40; greater than 40 years), and age at first pregnancy (less than 20; 20–25; 26–30; 31–35; 36–40; greater than 40 years).

Other outcomes

We will assess the proportion of women receiving appropriate screening and appropriate management separately at each visit for anemia, hypertension, and diabetes. We will also measure the retention rate from ANC to PPC and timely screening and management of fetal growth. The data entry in the eRegistry is continuous and the randomization permanent, which enable the analysis of overall health outcomes in a continuous and longitudinal manner for each individual woman. We might assess other outcomes if the intervention(s) shows an effect.

Timeline

Care providers in PHCs randomized to the TCC intervention have had the opportunity to register pregnant women to the text message service since June 2019. Care providers in PHC randomized to the QID intervention received access to the QID December 2019. We expect to reach the target sample size after approximately 6 months (Fig. 4).

Fig. 4 [Images not available. See PDF.]

Schedule of enrolment, interventions, and assessments (SPIRIT figure)

Sample size

Sample size calculations for the primary outcomes were performed in STATA “clustersampsi” (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP). We estimated the minimum detectable difference for comparison of each intervention arm with the control arm without assuming the effect of multiple comparison (using a single control to each intervention arm). The control prevalence for the primary outcome, using data from the eRegQual CRCT, ranged from 30 to 85% for comparison I (QID vs control) and from 34 to 53% for comparison II (TCC vs control) (unpublished results). The 138 eligible clusters have an average of 172 new pregnancies per 6 months (ranging from 23 to 1500) and a 1.69 coefficient of variation of cluster size (2016 data). We assumed a priori intra-cluster correlation coefficient (ICC) of 0.05 and set the statistical significance level to 5%. We are 80% powered to detect an estimated relative 50% increase for the outcomes with the smallest control prevalence (e.g., improving anemia screening and management at 24–28 weeks’ gestation from 30 to 45%).

Recruitment

PHCs that fulfilled the recruitment criteria were randomized to one of the four arms. Healthcare providers working in PHCs randomized to one or both of the interventions received training as described (the “Intervention” section).

No financial or non-financial incentives are or will be provided to the woman, public health officers, or healthcare providers at the PHCs included in the trial.

Allocation

Statisticians at the Center for Intervention Science in Maternal and Child Health (CISMAC), University of Bergen, Norway, performed the randomization independently without any influence from the research team.

PHCs were allocated to the TCC intervention, QID intervention, TCC and QID intervention, or control group with an equally 1:1:1:1 randomization ratio. The randomization was stratified by the point in time the eRegistry was implemented, and constrained on laboratory availability, ultrasound availability, and the size of the PHC.

In total, 10,000 randomization allocations were generated. The 10% best and balanced allocations with the least differences between the arms for the given covariates were identified, and one of these allocations were randomly selected for the trial [36].

Blinding

Healthcare providers and women attending care at PHCs included in the trial are blinded to the outcome measures. Women are expected to be blinded to the allocation of the QID intervention, but not to the TCC intervention due to the nature of the intervention. Healthcare providers are not blinded to the allocation of any intervention.

Data will be extracted from the eRegistry and transformed to anonymous datasets by blinded data analysts at PNIPH. Allocation codes will be generated for each primary outcome separately (e.g., A, B, C, and D; E, F, G, and H; I, J, K, L, and M). Our independent trial monitors (CISMAC) will keep the codes, which will be provided as allocation groups (intervention TCC, intervention QID, intervention TCC and QID vs control) for each set of outcomes after the completion of analyses.

Data extraction methods

Healthcare providers will routinely and continuously enter data in the eRegistry during patient care, and all outcomes are informed by secondary data. Anonymous data will be extracted from the eRegistry in accordance with the standard operating procedure (SOP) of the Palestinian maternal and child health eRegistry for routine registry operations and use of data for research purposes.

Data management

The data in the eRegistry will be managed in accordance with the governance structure approved by the Palestinian MoH. Only pre-defined anonymous data needed for the outcomes will be extracted from the eRegistry by PNIPH staff for this trial. The pre-defined anonymous original dataset will be provided to the trial sponsors and to our independent trial monitors (CISMAC), for independent monitoring and safeguarding.

Statistical methods

Intention-to-treat analysis will be performed for the primary outcomes to compare each independent intervention arms (QID, TCC, and QID and TCC) with the control, using individual-level data taking the design effect of the clustering into account. Descriptive statistics will be reported, and appropriate tests will be used to compare variables between the groups. Statistical significance will be set at p < 0.05. Appropriate bivariate and multivariate regression analyses will be performed. The regression analyses will take the design effect of the clustering into account and enable adjustment for any relevant variables not accounted for during randomization. We will do complete case analyses and consider appropriate imputations for missing data. We will present cluster-wise effects of the intervention to explore whether a disproportionately large part of the effect can be ascribed to extreme effects in a few large clusters. The interaction effect of the two interventions will be performed as a secondary analysis. We will conduct per-protocol analyses in cases of protocol violations, including withdrawal of the eRegistry, and data from these clusters will be excluded from the time of violation when appropriate. Data will be analyzed using the latest version of STATA (Stata Statistical Software: College Station, TX: StataCorp LP).

Data monitoring

Data management and monitoring will be done in accordance with the SOP of the Palestinian maternal and child health eRegistry for routine registry operations. We have not designated a Data and Safety Monitoring Committee, due to the lack of potential significant harm, nor an Endpoint Adjudication Committee, since we will not use any subjective clinical data for outcome measures.

Harms

This CRCT only utilizes the moment of opportunity of an ongoing implementation to study new digital interventions in a health systems approach. No potential for clinically significant harm has been identified during the development and implementation of this trial. Potential discomfort from worries may occur among women attending ANC in clusters receiving the TCC intervention, and this will be examined in a sub-sample (Fig. 1).

Confidentiality

Data confidentiality will be handled in accordance with the Palestinian MoH’s legal framework for maternal and child health electronic registries. This CRCT will only utilize anonymous data to enable the assessment of the effectiveness of the interventions. We will publish only aggregate data, and no data on individual clients, care providers, or identifiable clusters will be published.

Access to data

The data in the eRegistry belong to the Palestinian MoH, and the researchers will not have access to the entire registry or identifiable data of any kind. We will not publish the full data set as our legal rights to the data is limited to this analysis, and the richness of data would allow for several other analyses for which there is no ethics approval or approval from the Palestinian MoH. We will, however, publish syntaxes needed to recreate the data set from the eRegistry.

Dissemination plan

This protocol follows the Standard Protocol Items: Recommendations for Intervention trials (SPIRIT) guidelines. A formal revision of the protocol will be done if the change can affect the study’s nature. The sponsors and our independent trial monitors (CISMAC) will have to agree on the revision, which will require renewed approvals from both the ethic committee in Norway and in Palestine, and the Palestinian health authorities. We will inform all users and stakeholders and publish the results of the CRCT in peer-reviewed open-access journals according to the Consolidating Standards of Reporting Trials (CONSORT) guidelines and the mHealth Evidence Reporting and Assessment (mERA) checklist. Results will also be presented at scientific meetings and congresses and to the Palestinian MoH directly. With permission from the MoH, we will inform all participating PHC and their staff directly. We will acknowledge any change in the study outcomes, study design, sample sizes, or significant administrative aspects that will impact the study’s nature when disseminating the findings. Authorship will be in line with the recommendations of the International Committee of Medical Journal Editors. Summaries of the results and other relevant information will be published on the eRegistries website.

Discussion

We have designed a multi-arm CRCT with two digital health interventions, namely the QID for healthcare providers and TCC sent via SMS to pregnant women. The eRegistry employs the collection and use of systematic, uniform, and longitudinally entered routine clinical data, in algorithms that instantly drive these digital health interventions [5]. The QID and TCC interventions’ can impact the quality of the ANC service and the utilization of care by the pregnant woman, and the eRegCom CRCT aims to estimate the effectiveness.

Routine data from the eRegistry in Palestine demonstrates that both quality of care and utilization of care have significant room for improvement, and both need to be addressed to achieve a high effective coverage. To ensure healthy lives and promote well-being for all at all ages (Sustainable Development Goal 3), data and digital health are highlighted as accelerators [37]. Our interventions directly address the traditional lack of available clinical data in a timely manner from routine health information system to healthcare providers and clients in low- and middle-income countries [38], and the trial is a direct response to the WHO review group on digital health interventions’ request for effectiveness studies [19].

We used the Principles for Digital Development [39] and a human-centered design approach [40] to develop the eRegistry, and also the QID and TCC interventions. The Palestinian MoH and PNIPH have been heavily involved in all stages of the CRCT and led the communication towards healthcare providers. We will, in addition, assess the effect of the TCC intervention on women’s worries in pregnancy and their perceptions of ANC, to ensure that we do not introduce harm. Also, the unintentional exacerbation of inequities based on owning versus having access to a mobile phone will be explored. We do not have data to explore inequities towards those that do not own or have access to a mobile phone, but acknowledge this issue.

Our experience from the pilot data collection and a recent time-motion study [41] is that women are willing to participate, and we expect a high response rate in the assessment of women’s worries and perception of ANC. We anticipate to follow a comprehensive sample of women throughout ANC due to Palestinian’s restrictions on movement. Our large sample of PHCs, with no individual-level eligibility criteria, is representative for Palestine and presumably also for other settings with a similar organizational structure.

One or two healthcare providers per PHC is the most common organizational structure in Palestine, but some healthcare providers may work or be relievers in more than one clinic. Healthcare providers working in both TCC intervention and control PHCs will receive the TCC training, but they can only register women to receive text messages from the TCC intervention PHC. Healthcare providers working in both QID intervention and control PHCs will not receive the QID training, and therefore not the full intervention. This is to avoid contamination, as the action needed to improve care can be done from any PHC without the QID tool itself. Even though very few PHCs in the trial are operated by the same healthcare provider, this may potentially lead to an underestimation of the effectiveness.

Other limitations of this trial include the cadres of healthcare professions we target, which is only nurses, midwives, and community health workers. We exclude doctors, due to the risk of contamination as they work in several PHCs. However, we acknowledge that doctors play a significant role in the care for pregnant women. Our outcomes are dependent on the completeness of documentation by the healthcare provider, as our data source, the eRegistry, is the routine governmental documentation tool with few required data points. In order for the clusters to have comparable opportunities to succeed or fail, when it comes to management, we only include the first step in the management algorithm for our primary outcomes. For example, women with mild and moderate anemia, a new hemoglobin result after 4 weeks, will be counted as a success, without taking into account the response to that result. However, we will assess the total management chain in the secondary analysis.

We work closely with the Palestinian MoH to mitigate the risk of changes to the PHCs’ activities and staff during the trial. A key aspect of the work with eRegistries is to facilitate the uptake of the evidence-based findings in other countries and promote our digital interventions as “Global Public Goods” [29]. The eRegistry does involve high start-up costs, but once implemented, the intervention only requires moderate additional investments. We are in the midst of a digital revolution, and more than 68 countries are using DHIS2 [28], the eRegistry’s platform. The added value of the digital interventions can add value to investments in eRegistries.

Trial status

The CRCT began recruiting on 1 December 2019, and we expect to reach the target sample size after 6 months, approximately 1 June 2020. Protocol version 2—April 2020.

Acknowledgements

This CRCT is conducted in cooperation with the MoH in Palestine, district directors of health, district supervisors, obstetricians, doctors, nurses, and midwives, and the implementation team at the PNIPH and the Norwegian Institute of Public Health. We designed the study and its implementation, the intervention, and outcomes in close consultation with stakeholder groups in Palestine consisting of MoH officials, district directors of health, district supervisors, obstetricians, doctors, nurses, and midwives. The DHIS2 team, a part of the Health Information Systems Programme, Department of Informatics at the University of Oslo has been responsible for software customization and development. This trial is a part of the portfolio of CISMAC, which is a Centre for Excellence funded by the Research Council of Norway and the University of Bergen. Halvor Sommerfelt (director at CISMAC) and José Martines (scientific coordinator at CISMAC) provided guidance and advice for protocol development, while Hans Steinsland (statistician at University of Bergen) undertook the randomization.

Trial sponsor

The primary trial sponsor is the PNIPH, Al-Bireh P. O Box 4284, Palestine, and the secondary sponsor is the Norwegian Institute of Public Health, P. O Box 222 Skøyen, 0213 Oslo, Norway.

Authors’ contributions

KM, BB, JFF, BG, KF, MB, MJF, MV, YR, TH, and ZN contributed to the study design and formulation of outcomes. KM, BB, MV, and JFF contributed to the writing of the protocol and this manuscript. BB and MV contributed to sample size calculations. BG leads the implementation of the maternal and child health eRegistry in Palestine. MI, BO, EA, IAW, AA, SI, SEQ, KAK, and TA have contributed to the customization of the interventions and helped in providing relevant contextual knowledge and in all practical aspects of the CRCT. JFF has conceptualized the study, the intervention, outcomes, and data management strategies. BB and JFF undertook registration of the trial. All authors contributed to the refinement of this manuscript and approved the final version.

Funding

This research project is funded by the European Research Council (grant agreement number: 617639, Project title: A New Paradigm for Public Health Surveillance: Unlocking the Potential of Data to Empower Woman and Health Systems, Project acronym: HEALTMPOWR) and the Research Council of Norway (grant agreement number: 234376, Project title: Harmonized Reproductive Health Registry Communication Strategies: Using Health Data to Empower Women and Health Systems). It is also supported in kind by the Research Council of Norway (grant agreement number: 223269, Project title: Center for Intervention Science in Maternal and Child Health (CISMAC)); the University of Bergen, Norway; the Norwegian Institute of Public Health, Oslo, Norway; the World Health Organization through the Palestinian National Institute of Public Health, Ramallah, Palestine; and the Ministry of Health, Ramallah, Palestine. The funders had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Availability of data and materials

Not applicable.

Ethics approval and consent to participate

The study will be conducted according to the Palestinian MoH regulations and PNIPH research policies and procedures. All PHCs under study, intervention and control alike, have been notified by the MoH in Palestine about the research. Women are not required to give informed consent to be part of the health information system—the eRegistry, but they have to agree to receive text messages. Women in the sub-sample, to assess worries and perceptions of ANC, are required to give informed consent. This trial is considered non-medical health systems research and uses anonymous health data. Individuals are not regarded as research participants as all treatment decisions remain the responsibility of the healthcare professionals, and are not determined by the trial allocation [42]. The study protocols for the eRegCom trial and the sub-sample data collection have been approved by the ethics committee in Palestine, the Palestinian Health Research Council (401/18 and 670/19). The trial has received exemption from review by the Regional Committee for Health Research Ethics - Section South East B, Norway (2018/1148C).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Abbreviations

ANC

Antenatal Care

CISMAC

Center for Intervention Science in Maternal and Child Health

CRCT

Cluster Randomized Controlled Trial

DHIS2

District Health Information Software 2

MCH

Maternal and Child Health

MoH

Ministry of Health

NGO

Non-Governmental Organizations

PHC

Primary Healthcare Clinic

PNIPH

Palestinian National Institute of Public Health

PPC

Postpartum Care

QID

Quality Improvement Dashboard

SOP

Standard Operating Procedure

TCC

Targeted Client Communication

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s13063-020-04980-1.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Koblinsky, M; Moyer, CA; Calvert, C; Campbell, J; Campbell, OM; Feigl, AB; Graham, WJ; Hatt, L; Hodgins, S; Matthews, Z et al. Quality maternity care for every woman, everywhere: a call to action. Lancet; 2016; 388, 10057 pp. 2307-2320. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27642018][DOI: https://dx.doi.org/10.1016/S0140-6736(16)31333-2]

2. WHO. The global strategy for women’s, children’s and adolescents’ health (2016-2030); 2015; Geneva, WHO:

3. Ng, M; Fullman, N; Dieleman, JL; Flaxman, AD; Murray, CJ; Lim, SS. Effective coverage: a metric for monitoring universal health coverage. PLoS Med; 2014; 11, 9 e1001730. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25243780][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4171091][DOI: https://dx.doi.org/10.1371/journal.pmed.1001730]

4. Shengelia, B; Tandon, A; Adams, OB; Murray, CJ. Access, utilization, quality, and effective coverage: an integrated conceptual framework and measurement strategy. Soc Sci Med; 2005; 61, 1 pp. 97-109. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/15847965][DOI: https://dx.doi.org/10.1016/j.socscimed.2004.11.055]

5. Froen, JF; Myhre, SL; Frost, MJ; Chou, D; Mehl, G; Say, L; Cheng, S; Fjeldheim, I; Friberg, IK; French, S et al. eRegistries: electronic registries for maternal and child health. BMC Pregnancy Childbirth; 2016; 16, 1 11. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26791790][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4721069][DOI: https://dx.doi.org/10.1186/s12884-016-0801-7]

6. Kreps, GL; Neuhauser, L. New directions in eHealth communication: opportunities and challenges. Patient Educ Couns; 2010; 78, 3 pp. 329-336. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20202779][DOI: https://dx.doi.org/10.1016/j.pec.2010.01.013]

7. Free, C; Phillips, G; Watson, L; Galli, L; Felix, L; Edwards, P; Patel, V; Haines, A. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med; 2013; 10, 1 e1001363. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23458994][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3566926][DOI: https://dx.doi.org/10.1371/journal.pmed.1001363]

8. Norman, GJ. Answering the “what works?”. Question in health behavior change. Am J Prev Med; 2008; 34, 5 pp. 449-450. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18407014][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2405809][DOI: https://dx.doi.org/10.1016/j.amepre.2008.02.005]

9. Ivers, N; Jamtvedt, G; Flottorp, S; Young, JM; Odgaard-Jensen, J; French, SD; O’Brien, MA; Johansen, M; Grimshaw, J; Oxman, AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev; 2012; 6, Cd000259.

10. Kluger, A; DeNisi, A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull; 1996; 119, 2 pp. 254-284. [DOI: https://dx.doi.org/10.1037/0033-2909.119.2.254]

11. Jamtvedt, G; Young, JM; Kristoffersen, DT; O’Brien, MA; Oxman, AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev; 2006; 2, CD000259.

12. Ivers, NM; Grimshaw, JM; Jamtvedt, G; Flottorp, S; O’Brien, MA; French, SD; Young, J; Odgaard-Jensen, J. Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med; 2014; 29, 11 pp. 1534-1541. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24965281][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4238192][DOI: https://dx.doi.org/10.1007/s11606-014-2913-y]

13. Hysong, SJ; Best, RG; Pugh, JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implementation Sci; 2006; 1, 9. [DOI: https://dx.doi.org/10.1186/1748-5908-1-9]

14. Larson, EL; Patel, SJ; Evans, D; Saiman, L. Feedback as a strategy to change behaviour: the devil is in the details. J Eval Clin Pract; 2013; 19, 2 pp. 230-234. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22128773][DOI: https://dx.doi.org/10.1111/j.1365-2753.2011.01801.x]

15. Lee, SH; Nurmatov, UB; Nwaru, BI; Mukherjee, M; Grant, L; Pagliari, C. Effectiveness of mHealth interventions for maternal, newborn and child health in low- and middle-income countries: systematic review and meta-analysis. J Glob Health; 2016; 6, 1 010401. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26649177][DOI: https://dx.doi.org/10.7189/jogh.06.010401]

16. Tomlinson, M; Rotheram-Borus, MJ; Swartz, L; Tsai, AC. Scaling up mHealth: where is the evidence?. PLoS Med; 2013; 10, 2 e1001382. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23424286][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3570540][DOI: https://dx.doi.org/10.1371/journal.pmed.1001382]

17. Marcolino, MS; Oliveira, JAQ; D'Agostino, M; Ribeiro, AL; Alkmim, MBM; Novillo-Ortiz, D. The impact of mHealth interventions: systematic review of systematic reviews. JMIR mHealth uHealth; 2018; 6, 1 e23. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29343463][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5792697][DOI: https://dx.doi.org/10.2196/mhealth.8873]

18. Labrique, AB; Vasudevan, L; Kochi, E; Fabricant, R; Mehl, G. mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Glob Health Sci Pract; 2013; 1, 2 pp. 160-171. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25276529][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4168567][DOI: https://dx.doi.org/10.9745/GHSP-D-13-00031]

19. WHO. WHO guideline: recommendations on digital interventions for health system strengthening; 2019; Geneva, World Health Organization:

20. Fjeldsoe, BS; Marshall, AL; Miller, YD. Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med; 2009; 36, 2 pp. 165-173. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19135907][DOI: https://dx.doi.org/10.1016/j.amepre.2008.09.040]

21. Chen, H; Chai, Y; Dong, L; Niu, W; Zhang, P. Effectiveness and appropriateness of mHealth interventions for maternal and child health: systematic review. JMIR mHealth uHealth; 2018; 6, 1 e7. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29317380][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5780618][DOI: https://dx.doi.org/10.2196/mhealth.8998]

22. Sondaal, SF; Browne, JL; Amoakoh-Coleman, M; Borgstein, A; Miltenburg, AS; Verwijs, M; Klipstein-Grobusch, K. Assessing the effect of mHealth interventions in improving maternal and neonatal care in low- and middle-income countries: a systematic review. PLoS One; 2016; 11, 5 e0154664. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27144393][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4856298][DOI: https://dx.doi.org/10.1371/journal.pone.0154664]

23. Gurol-Urganci, Ipek, et al. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database Syst Rev. 12 (2013). https://scholar.google.com/scholar?hl=no&as_sdt=0%2C5&q=Gurol-Urganci+I%2C+de+Jongh+T%2C+Vodopivec-Jamsek+V%2C+Atun+R%2C+Car+J.+Mobile+phone+messaging+reminders+for+attendance+at+healthcare+appointments.+Cochrane+Database+Syst+Rev.+2013%3B%2812%29.&btnG=.

24. Statistics PCBo. Children–issues and statistics, child statistics series (no. 16). Annual Report Palestine 2013.

25. Venkateswaran, M; Morkrid, K; Ghanem, B; Abbas, E; Abuward, I; Baniode, M; Norheim, OF; Froen, JF. eRegQual-an electronic health registry with interactive checklists and clinical decision support for improving quality of antenatal care: study protocol for a cluster randomized trial. Trials; 2018; 19, 1 54. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29357912][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5778657][DOI: https://dx.doi.org/10.1186/s13063-017-2386-5]

26. Venkateswaran, M; Bogale, B; Abu Khader, K; Awwad, T; Friberg, IK; Ghanem, B; Hijaz, T; Morkrid, K; Froen, JF. Effective coverage of essential antenatal care interventions: a cross-sectional study of public primary healthcare clinics in the West Bank. PLoS One; 2019; 14, 2 e0212635.[COI: 1:CAS:528:DC%2BC1MXmtlGqsL4%3D] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30794645][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6386267][DOI: https://dx.doi.org/10.1371/journal.pone.0212635]

27. Frost, KS. Designing a theory-based digital audit and feedback dashboard for maternal and child health care providers: a qualitative study of preferences and needs in Palestine; 2019;

28. DHIS 2 [https://www.dhis2.org/].

29. The eRegistries Initiative [www.eregistries.org].

30. Giacaman, R; Abu-Rmeileh, NME; Wick, L. The limitations on choice: Palestinian women’s childbirth location, dissatisfaction with the place of birth and determinants. Eur J Public Health; 2007; 17, pp. 86-91. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16798783][DOI: https://dx.doi.org/10.1093/eurpub/ckl089]

31. Rahim, HFA; Wick, L; Halileh, S; Hassan-Bitar, S; Chekir, H; Watt, G; Khawaja, M. Maternal and child health in the occupied Palestinian territory. Lancet; 2009; 373, 9667 pp. 967-977. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19268353][DOI: https://dx.doi.org/10.1016/S0140-6736(09)60108-2]

32. Gude, WT; Brown, B; van der Veer, SN; Colquhoun, HL; Ivers, NM; Brehaut, JC; Landis-Lewis, Z; Armitage, CJ; de Keizer, NF; Peek, N. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implementation Sci; 2019; 14, 1 39. [DOI: https://dx.doi.org/10.1186/s13012-019-0887-1]

33. Keller, PA; Harlam, B; Loewenstein, G Volpp KGJJoCp. Enhanced active choice: a new method to motivate behavior change. J Consum Psychol; 2011; 21, 4 pp. 376-383. [DOI: https://dx.doi.org/10.1016/j.jcps.2011.06.003]

34. Langley, GJ; Nolan, KM Nolan TWJQP. The foundation of improvement. Qual Prog; 1994; 27, 6 pp. 81-86.

35. Bogale, B; Morkrid, K; O'Donnell, B; Ghanem, B; Abu Ward, I; Abu Khader, K; Isbeih, M; Frost, M; Baniode, M; Hijaz, T et al. Development of a targeted client communication intervention to women using an electronic maternal and child health registry: a qualitative study. BMC Med Inform Decis Mak; 2020; 20, 1 1. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31906929][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6945530][DOI: https://dx.doi.org/10.1186/s12911-019-1002-x]

36. Dickinson, LM; Beaty, B; Fox, C; Pace, W; Dickinson, WP; Emsermann, C; Kempe, A. Pragmatic cluster randomized trials using covariate constrained randomization: a method for practice-based research networks (PBRNs). J Am Board Fam Med; 2015; 28, 5 pp. 663-672. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26355139][DOI: https://dx.doi.org/10.3122/jabfm.2015.05.150001]

37. WHO. Towards a global action plan for healthy lives and well-being for all: uniting to accelerate progress towards the health-related SDGs; 2018; Geneva, World Health Organization:

38. Lippeveld, T. Routine health facility and community information systems: creating an information use culture. Glob Health Sci Pract; 2017; 5, 3 pp. 338-340. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28963169][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5620331][DOI: https://dx.doi.org/10.9745/GHSP-D-17-00319]

39. Principles for Digital Development [https://digitalprinciples.org/].

40. Rinkus, S; Walji, M; Johnson-Throop, KA; Malin, JT; Turley, JP; Smith, JW; Zhang, J. Human-centered design of a distributed knowledge management system. J Biomed Inform; 2005; 38, 1 pp. 4-17. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/15694881][DOI: https://dx.doi.org/10.1016/j.jbi.2004.11.014]

41. Lindberg, MH; Venkateswaran, M; Abu Khader, K; Awwad, T; Ghanem, B; Hijaz, T; Morkrid, K; Froen, JF. eRegTime, efficiency of health information management using an electronic registry for maternal and child health: protocol for a time-motion study in a cluster randomized trial. JMIR Res Protocols; 2019; 8, 8 e13653. [DOI: https://dx.doi.org/10.2196/13653]

42. Weijer, C; Grimshaw, JM; Eccles, MP; McRae, AD; White, A; Brehaut, JC; Taljaard, M. The Ottawa statement on the ethical design and conduct of cluster randomized trials. PLoS Med; 2012; 9, 11 e1001346. [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23185138][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3502500][DOI: https://dx.doi.org/10.1371/journal.pmed.1001346]