Impact of using eHealth tools to extend health services to rural areas of Nigeria: protocol for a mixed-method, non-randomised cluster trial

Abstract

Introduction

eHealth solutions that use internet and related technologies to deliver and enhance health services and information are emerging as novel approaches to support healthcare delivery in sub-Saharan Africa. Using digital technology in this way can support cost-effectiveness of care delivery and extend the reach of services to remote locations. Despite the burgeoning literature on eHealth approaches, little is known about the effectiveness of eHealth tools for improving the quality and efficiency of health systems functions or client outcomes in resource-limited countries. eHealth tools including satellite communications are currently being implemented at scale, to extend health services to rural areas of Nigeria, in Ondo and Kano States and the Federal Capital Territory. This paper shares the protocol for a 2-year project (‘EXTEND’) that aims to evaluate the impact of eHealth tools on health system functions and health outcomes.

Methodology and analysis

This multisite, mixed-method evaluation includes a non-randomised, cluster trial design. The study comprises three phases—baseline, midline and endline evaluations—that involve: (1) process evaluation of video training and digitisation of health data interventions; (2) evaluation of contextual influences on the implementation of interventions; and (3) impact evaluation of results of the project. A convergent mixed-method model will be adopted to allow integration of quantitative and qualitative findings to achieve study objectives. Multiple quantitative and qualitative datasets will be repeatedly analysed and triangulated to facilitate better understanding of impact of eHealth tools on health worker knowledge, quality and efficiency of health systems and client outcomes.

Ethics and dissemination

Ethics approvals were obtained from the University of Leeds and three States’ Ministries of Health in Nigeria. All data collected for this study will be anonymised and reports will not contain information that could identify respondents. Study findings will be presented to Ministries of Health at scientific conferences and published in peer-reviewed journals.

Trial registration number

ISRCTN32105372; Pre-results.

Correspondence to Dr Matthew John Allsop; [email protected]

Strengths and limitations of this study

• This multidisciplinary, mixed-method study, including a non-randomised cluster trial, will shed light on how the processes and context of implementation of eHealth tools influence improvements in health systems function and client outcomes.

• Our focus on extending basic services to hardest-to-reach clients will assess the usefulness of eHealth tools in contributing to universal health coverage.

• The relatively short duration of this initial study could limit our ability to assess the impact(s) of the project on health systems functions and health outcomes in Nigeria, though short-term outcomes will be observable.

• The quantitative design limitations (eg, non-randomised trial) means that our study will not be able to attribute causation, and any intervention effect estimates will be at risk from a range of biases.

Introduction 

Health systems challenges in Nigeria include chronic infrastructure deficits, weak and irregular staff training, and deficient data management. These challenges severely affect healthcare delivery. eHealth approaches, using internet and related information and communication technologies (ICTs) to deliver and enhance health services and information,^1–3 are emerging as novel approaches to support healthcare delivery in sub-Saharan Africa⁴ including maternal, newborn and child health (MNCH) services. Using digital technology in this way can improve cost-effectiveness of care delivery^5–7 and extend the reach of services to remote locations.^{8 9} Nigeria has been slow at adopting eHealth approaches,¹⁰ due in part, to the cost of providing mobile network infrastructure in rural areas,¹¹ inadequate road networks and increased investment risk arising from security concerns. Consequently, only 87% of Nigeria’s population has access to 2G network coverage, and 51% have access to 3G coverage,¹² thus limiting opportunities for eHealth approaches for healthcare delivery.¹² An approach to overcoming such limited connectivity that will enable policy-makers to extend the reach of healthcare services to populations in rural areas is the use of satellite communication (SatCom)¹³ to provide communication links with no need for physical infrastructure, that is, mast and cables.¹⁴ The EXTEND project in Nigeria seeks to address logistic and technical challenges of providing care to those hardest to reach (the so-called last mile challenge), by using satellite technology to extend communications infrastructure to rural areas. This is anticipated to improve the standards of MNCH services, contributing to addressing the Sustainable Development Goal 3 of ensuring healthy lives and promoting well-being for all people of ages.¹⁵

Many studies on eHealth are criticised for being pilot studies with small sample sizes that rely on qualitative assessment designs^{16 17} and for providing minimal information about the effectiveness of eHealth tools for improving quality and efficiency of health systems functions and/or client outcomes.¹⁸ To better understand the impact of eHealth projects, scholars now recommend the adoption of multidimensional evaluation approaches that use mixed-method designs^{16 17} with larger sample sizes to examine the effects of such programmes on providers, clients and on health systems. The EXTEND project therefore adopted a rigorous mixed-method approach to evaluate scale-up of eHealth interventions to technologically disadvantaged areas across three states of Nigeria, that is, Kano and Ondo States and the Federal Capital Territory (FCT). The interventions are explained shortly.

The project represents an international multisectorial partnership that includes: (1) a global SatComs company (Inmarsat Global Limited), (2) a Nigerian mobile health implementation company (InStrat Global Health Solutions or ‘InStrat’ for short), (3) four academic institutions (the University of Leeds in the UK, and Bayero University Kano, the University of Abuja, and the University of Lagos in Nigeria), (4) the Federal Ministry of Health and State Ministries of Health in Ondo, Kano and the FCT Department of Health (DOH).

The project aims to understand whether eHealth tools lead to benefits and under what circumstances using SatCom to extend health services to remote areas contributes to improved health systems functions and health outcomes. Specific objectives are to:

• Strengthen service delivery through enabling access to a video training (VTR) app that targets knowledge and skills, with at least 65% of frontline health workers (FHWs) showing improvements between pretest and post-test assessments.

• Strengthen data management using the Clinical Patient Administration Kit (CliniPAK) app to enable at least 90% participating primary healthcare (PHC) facilities to transmit accurate and timely data to local government area (LGA) headquarters.

• Identify factors that influence the acceptability and use of VTR and CliniPAK at scale for FHWs.

The purpose of this paper is to share the study protocol for evaluating the impact of eHealth tools for extending basic health services to remote areas in Nigeria. As there are no widely used systems for disseminating eHealth protocols or reporting non-randomised cluster trials, we will draw on different checklists for reporting empirical results of our work. These include the Consolidated Standards of Reporting Trials checklist for reporting trials and a recently published mHealth Evidence Reporting and Assessment checklist for improving comprehensiveness and quality of digital health evidence.¹⁹ In this protocol, we outline the study design and methods including study setting, conceptual framework, data collection and analysis methods. We also explain key ethics and research governance issues, and our approach to dissemination.

Study design and methods

Study setting and target population

The eHealth interventions will be implemented by ‘InStrat’ from March 2017 to March 2019, in collaboration with the State Ministries of Health in Ondo and Kano States and the DOH in the FCT. A successful pilot testing of VTR and CliniPAK apps in Ondo State in 2016 led to scaling up of eHealth interventions to Kano state and the FCT in 2017. In this evaluation study, we have selected two clusters in each state corresponding to LGAs: one LGA with facilities implementing VTR and CliniPAK tools, and the other LGA with facilities not implementing any eHealth intervention. The ‘intervention’ LGAs will be assessed against non-intervention LGAs. Intervention LGAs (see table 1) were selected because they had many primary healthcare facilities situated in areas without access to regular mobile network service.

Table 1

Intervention and control local government areas (LGAs) selected by state

Participating state	Intervention LGAs	No of intervention facilities	Modes of delivery of eHealth tools, and No of facilities using each mode	Non-intervention LGAs (all local network)	No of non-intervention facilities
Federal Capital Territory	1. Gwagwalada	29	SatCom 3 local network 26	1. Kuje	29
Kano State	1. Dawakin Tofa 2. Sumaila	35	SatCom 35 local network 0	2. Garun Mallam	26
Ondo State	1. Akoko South,	62	SatCom 37 local network 25	1. Irele	21
	2. Idanré,			2. Ondo East	24
	3. Odigbo			3. Akoko Northwest	25
Summary	Total intervention LGAs=6	Total intervention facilities=126	Total SatCom=75 Total local network=51	Total control LGAs=5	Total control facilities=125

SatCom, satellite communication.

A total of 126 PHC facilities in intervention LGAs across the three states have, since April 2017, been incrementally supplied with tablet computers loaded with data plans to enable the VTR and CliniPAK interventions. Health workers in these PHC facilities were then trained by InStrat staff to use the tablets. See table 2 for a description of VTR and CliniPAK interventions. Moreover, 75 SatCom facilities in intervention LGAs will be supplied with a broadband global area network link-based SatCom hardware, to enable internet connectivity in the PHC facilities. The remaining 51 non-SatCom facilities in intervention LGAs are already connected via regular terrestrial mobile network operators and so do not require linking via broadband global area network link-based SatCom hardware. Beyond the training to enable staff to use the tablets, InStrat staff will provide ongoing technical support to ensure that SatCom and tablets continue to function and that FHWs capacity is maintained despite attrition.

Table 2

Overview of eHealth tools

eHealth tool	Description of tool
Clinical Patient Administration Kit (CliniPAK)	A tablet computer-enabled point-of-care data capture and decision support tool that allows FHWs to capture patient health information and send appropriate data to remote servers through mobile networks. The CliniPAK software provides an electronic medical record that incorporates data on patient registration and demographics, vital signs, diagnosis, treatment, case review and administrative task support. The software triggers immediate alerts for at-risk patients, referrals to secondary health systems and on-demand reporting to enable health administrators increase productivity and improve patient clinical experience. CliniPAK was developed and is owned by Vecna Cares Charitable Trust, Cambridge, Massachusetts, USA.
Video training (VTR) application	The VTR education intervention consists of a series of videos adapted from the ‘ORB platform’ (www.health-orb.org/), with a set of quizzes administered via a derivative of the open source application—OppiaMobile App on tablet computers developed to test the users’ understanding of the training content. The intervention will be delivered to FHWs via a structured programme of bite-size training films addressing knowledge and skills requirements of FHWs concerning antenatal care, basic obstetric care, perinatal care and postnatal care. Relevant video content included in the training package was selected in consultation with State Ministries of Health. Installed on the tablet computers held at PHC facilities, the VTR package will provide high-quality learning for FHWs, by delivering clear, engaging clinical scenarios and educational messages for motivating FHWs who lack basic resources to support their work.27

FHW, frontline health worker; PHC, primary healthcare.

The target population for this evaluation study comprise three groups: (1) FHWs and facility heads at intervention PHC facilities; (2) pregnant women at participating PHC facilities and (3) policy-makers. The FHWs will include nurses, midwives, laboratory technicians and community health workers.

Study design

The study will use a mixed-method design to evaluate the acceptability and effects of novel eHealth tools implemented across three states of Nigeria. The quantitative part of the study will use a non-randomised cluster trial design, collecting longitudinal data before and after the implementation of eHealth tools in intervention facilities to compare with longitudinal National Health Management Information System (NHMIS) data in non-intervention facilities to understand the impact of the eHealth tools on health systems functions and health outcomes. The quantitative arm will also assess the impact of eHealth tools on FHWs’ knowledge in intervention sites only. The qualitative part of the study will enable: (1) process and impact evaluations of satellite connectivity and the scaled-up VTR and CliniPAK interventions in the three states; (2) evaluation of the influence of contextual factors on the implementation of the interventions. Implementation in states from different regions of Nigeria (Ondo in west, FCT in middle belt and Kano in the north) facilitates the examination of different contextual factors that may affect implementation and project outcomes.

Conceptual framework

To assess the relationship between project inputs, processes of implementation and outcomes, we will use the framework in figure 1 to conceptually explore how inputs lead to processes, how processes lead to outputs and how outputs contribute to outcomes and impact.²⁰ Given the significance of context to attaining project results, we will examine the roles of SatCom, VTR and CliniPAK interventions in achieving project effects within a wider context, rather than ascribing changes in results and outcomes to our project alone. To achieve this, we will use insight from analysis of documents review and qualitative interviews (see the ‘Methods of data collection’ section) to assess whether/how the ‘context of implementation’ of the project affects project results. For example, though figure 1 depicts linear and simplified relationships between inputs, activities, outputs and outcomes of the project, we acknowledge that the study findings can be influenced by competing/concurrent maternal and child health interventions in either the intervention or control arm of the study (or both) that were unknown or unanticipated at the time of developing the protocol. We also acknowledge that there can be unintended positive or negative consequences of our interventions that are not currently mentioned in this protocol.

Figure 1. Conceptual framework for EXTEND project, Nigeria. CliniPAK, Clinical Patient Administration Kit; HWs, health workers; PHC, primary healthcare; VTR, video training.

Recruiting FHWs for CliniPAK and VTR interventions

Three hundred FHWs in 126 intervention sites will be selected to participate in CliniPAK and VTR interventions based on lessons from pilot testing in 2016, alongside resource and logistical feasibility considerations. This will comprise 200 FHWs in 75 SatCom facilities and 100 FHWs in 51 non-SatCom facilities (ie, three from each SatCom facility and two from each mobile network-enabled facility). As part of their orientation, the objectives of the EXTEND project will be explained to FHWs in intervention sites. To minimise possibility of coercion, FHWs at intervention sites will then be approached by a member of the research team a week after their orientation and invited to participate in the project. FHWs who agree to participate in the project will be trained to use the CliniPAK app for the daily documentation of MNCH care. For VTR intervention, participants will be provided with login instructions for completing a pretutorial survey, reviewing an electronic tutorial (see table 2), and completing a post-tutorial survey. Participants will complete a consent form prior to participation. The project plans to provide 4–6 monthly refresher of VTR modules to encourage FHW retention in the study.

Methods of data collection and sampling

The evaluation will comprise three phases: baseline assessment within 3 months of start of project, midline assessment at 12 months (March 2018) and endline evaluation at 24 months (March 2019).

Phase 1: baseline assessment

Baseline assessment was conducted from 23 May to 30 June 2017 in intervention and control sites to ascertain the status of target key performance indicators (KPIs), before full implementation of the project, and involved assessment of three types of data:

• Historical NHMIS data from January to December 2016 were collected comprising numbers of pregnant women attending antenatal care (ANC), numbers delivering in health facilities and attended to by health professionals and numbers of women accessing postnatal care (PNC). The quality (completeness and accuracy) and indicator levels of these data were checked.

• In-depth interviews (IDIs) with 11 policy-makers and 31 facility heads, identified using purposive sampling, to ascertain how facilities used tools and will generate and transmit health service data to the NHMIS. Interviewees were also asked about contextual factors that could affect project implementation or its results.

• Qualitative interviews with 31 patients, selected through convenience sampling, to understand their motivation for using health services in the chosen PHC facilities; and their perception of standards of service in the PHC facilities.

Phases 2 and 3: midline and endline assessments

These phases will compare results with the baseline assessment. During each of these phases, a multidimensional approach will be adopted comprising:

• Document review of published literature, ICT and reproductive health policies, and contextual factors that may influence implementation of eHealth innovations.

• Assessment of changes in completeness and levels of NHMIS indicators for the trial, and changes in FHWs’ understanding of ANC, basic obstetric care, PNC and family planning using output data from pretutorial and post-tutorial surveys completed using the VTR.

• IDIs with 24 FHWs, 24 facility heads and 9 policy-makers, purposefully selected and asked about effectiveness and benefits of eHealth innovations for strengthening FHW understanding of MNCH and improving health systems functions.

• IDIs with 15 service users about their perception of quality of care following implementation of eHealth tools.

A project plan is shown in figure 2. All data collection and analysis during baseline, midline and endline assessments will be done by in-country university partners. Following baseline assessment, the three Nigerian universities produced state-level reports integrating quantitative and qualitative findings for the three states and these formed the dataset for a country baseline report. This approach to reporting will be repeated for midline and endline evaluations, respectively, to make sense of the effects and impacts of eHealth interventions.

Figure 2. Project work plan. CliniPAK, Clinical Patient Administration Kit; FHWs, frontline health workers; KPIs, key performance indicators; MNCH, maternal, newborn and child health; PHC, primary healthcare; SatCom, satellite communication; VTR, video training.

Additionally, a routine monitoring exercise, led by the University of Leeds, will run alongside the three phases of the study to provide quality assurance for the study. Interim evaluation of data quality (completeness and accuracy) will occur following monthly collation of routine NHMIS data alongside data from CliniPAK and VTR, collected by in-country teams. Monthly data will be collated into quarterly reports and submitted in aggregate form for audit by the University of Leeds monitoring team. This will support identification of discrepancies or irregularities in reporting and facilitate periodic performance reviews against KPIs. While monitoring will occur independently, findings from interim evaluations will be shared with study sponsors to inform project monitoring. To facilitate monitoring and ensure standardisation and consistency of reporting across the three states, a project logical framework (logframe) has been developed, outlining the project’s targets, indicators and means of verification of data collected by PHC facilities in the three states, to track progress towards meeting outputs, outcomes and potential impacts of the projects (see online supplementary appendix 1). In addition to the logframe, we have developed a defined set of KPIs to measure performance of against operational criteria (see online supplementary appendix 2). The KPIs will be monitored through periodic performance reviews and within baseline, midline and endline evaluation.

Trial outcomes

The primary outcome for the trial is a binary facility-level indicator measuring whether the monthly NHMIS indicator ‘total number of ANC visits’ is complete (ie, available through the NHMIS) for every month of the 6-month postintervention period. The secondary outcomes are: (1) binary facility-level indicators of whether the monthly NHMIS indicators ‘total PNC visits’ and ‘percentage skilled birth attendance’ are complete or not for every month of the 6-month postintervention period; and (2) the NHMIS indicators ‘total number of ANC visits’, ‘total number of PNC visits’ and ‘percentage of skilled birth attendance’.

Data analysis

For the non-randomised trial based on available resources, we will have 6 clusters in the intervention arm and 5 in the control arm, having a mean cluster size (number of facilities) of 25 and a cluster-size variance of 23. Based on preintervention data, for the primary outcome, we assumed an existing proportion in both arms of 0.18 and an intracluster correlation coefficient of 0.025. Using two-tailed testing at the 5% significance level, this allows us to detect an absolute reduction in the intervention arm to ≤0.01 with >80% power.²¹

We will analyse the primary outcome, adjusted for covariates, using a two-stage method that accounts for between-cluster variation and is appropriate for cluster trials with relatively few clusters per arm.²¹ First, we will use a logistic regression model of the primary outcome including our covariates of interest, but excluding the treatment effect, to compute a difference residual for each cluster. Second, we will estimate the intervention effect as the absolute difference in the primary outcome (intervention minus control), and base our inference on the associated (t-statistic based) 95% CIs and p value (two-sided, 5% level of significance). We will analyse all secondary outcomes related to NHMIS indicator data completeness using the same methods. All results will be adjusted for the baseline level of the relevant outcome, calculated as the facility-level proportion of data completeness for the monthly relevant outcome as collected over the 12 months prior to the implementation of the intervention, and for LGA.

We will use controlled interrupted time-series analysis to analyse whether there have been any changes in the levels and/or trends of all NHMIS indicators following implementation of the intervention. For all these monthly indicators, we will have 12 months’ worth of preintervention data and 6 months’ worth of postintervention data for both intervention and control clusters. We will analyse all NHMIS indicators, aggregated at the LGA level, using a linear regression model including a time × treatment × period (preintervention vs postintervention) interaction to provide estimates of the changes in level and trend of outcomes before and after the intervention period. If model errors display non-negligible autocorrelation, this will be accounted for using by fitting a generalised least squares model adjusting for AR(1) errors.

The models will include a random intercept for individual, potentially nested within a random intercept for facility if necessary. We will estimate the mean change in knowledge score percentage points based on the coefficient for a fixed effect of test time (post-test vs pretest). We will also control for a range of likely influential and potentially confounding covariates: age, sex, staff level (community health extension worker or nurse/midwife), facility type (basic or comprehensive) and state (FCT, Kano or Ondo). We will also explore whether any changes in knowledge scores differ between the following subgroups: (1) FHWs at SatCom vs non-SatCom sites, (2) FHWs at basic versus comprehensive facilities, (3) community health extension workers versus nurses/midwives, and (4) male versus female FHWs. We will again use linear mixed models (including the above covariates) to analyse changes in knowledge scores for each subgroup, and separate linear mixed models (including the above covariates) with an interaction between test time and the relevant subgroup indicator variable to provide estimates of any differences in change in knowledge scores between the subgroup comparisons listed. All inferences will be based on the associated (t-statistic based) 95% CI and two-sided p value (5% significance level) for the relevant coefficients.

During each phase of the project, IDIs with policy-makers, facility heads, FHWs and service users will be audio recorded (subject to informed consent), transcribed and where appropriate translated into English for manual data analysis. Framework approach will be used for understanding the impact of eHealth interventions on health system functions, while allowing for emergence of new themes. The framework approach includes the stages of familiarisation with data, coding, indexing and charting, mapping and interpretation of data.²²

Quantitative and qualitative findings will be integrated and triangulated to answer the research questions. Furthermore, we will conduct a comparative analysis of variations in adoption and effectiveness of eHealth innovations in the three states to ascertain the influence of contextual factors on processes of implementation and project outcomes. The two datasets will be repeatedly triangulated especially during the midline and endline evaluations to understand the impact of interventions on health systems functions and health outcomes.

Research governance

The project will be conducted with full respect for relevant legislations (eg, the Charter of Fundamental Rights of the European Union) and international conventions (eg, Helsinki Declaration). Data collection and analysis will take account of four key issues:

Protecting privacy and confidentiality of information collected from participants

The University of Leeds team will compile and analyse data collected by university teams in Nigeria and support their training, including providing information on protocols for anonymising and securely sharing study data. Data will be shared using online secure portals and will be stored with passwords and access only made available to data for those directly involved in data analysis. All transcripts from the study will be anonymised prior to sharing with the Leeds team.

Ensuring anonymity of participants

We will preserve the anonymity of study participants at all times. Unnecessary collection of personal data will be avoided, and respondents will have the right to review outputs and withdraw consent. Where personal data are collected (eg, age, sex, level of education), it will be coded, removed from the data for analysis and stored separately. Only designated project staff will have access to the keys linking the data with the personal information.

Maintaining independence of judgement

We will routinely review the independence of the research team when undertaking monitoring work. While working closely with partners in the consortium, we will ensure that we are free of influence over the judgements relating to the evaluation.

Avoiding bias and being fair

A comprehensive evaluation framework has been developed to direct data collection in the study. The project will also develop a shared online platform to facilitate data capture and reporting of variables for monitoring KPIs across project sites. The framework is impartial to any group and inclusive of all groups.

The project will be implemented according to standard governance practice at the University of Leeds for the implementation of collaborative projects. This includes ensuring regular communication between partners and engagement with policy-makers and practitioners; quality assurance through regular peer-review both within and between teams; appropriate mentoring and coaching support of junior researchers.

Communication and dissemination of results

Improving MNCH knowledge and practice is a national and international priority. This initial scale-up of eHealth interventions to the FCT, Ondo and Kano states will be further expanded to other states of Nigeria and to non-health sectors (education, agriculture and civic identity management). The high demand for this study from policy-makers and funders provides an excellent opportunity to ensure uptake of high-quality evidence into policy and practice. Specific methods of communicating study findings include a combination of the following:

• Developing newsletters, press releases to communicate key project findings in simple ways to the general public.

• Developing a dedicated website for the study where results will be publicly accessible by national and international policy-makers, practitioners and academics.

• Delivering presentations at national and international conferences and publishing articles in peer-reviewed journals with emphasis on open access where feasible.

We will ‘embed’ the research strategy development and assessment into policy and practice, working with the Federal Ministry of Health and State Ministries of Health in Ondo and Kano States and the DOH in the FCT. This embedded approach, developed by the Nuffield Centre of the University of Leeds, has been used in many countries to improve the quality and effectiveness of scaled-up programmes.^23–25 We will engage decision makers throughout the process in a research-policy partnership to facilitate adoption and scale-up of eHealth tools to other states in Nigeria.²⁶

Patient and public involvement

Patients were not involved in the development or design of the study. We will work with patient advocacy groups to ensure that plain language summaries of study findings are shared to both participating service users and wider patient groups.

Discussion

This paper reports a protocol for a mixed-method, non-randomised cluster trial of the use of eHealth tools for extending health services to rural areas in Nigeria. This multidisciplinary, mixed-method study aims to understand the role of eHealth approaches in improving the quality and efficiency of health systems functions and client outcomes. Since the start of the study, we have:

• Reviewed the project’s FHW training curriculum (March 2017), to align it with national and international MNCH guidelines for training FHWs.

• Conducted baseline assessment of key indicators (May–June 2017) to enable reliable comparison against findings of midline and endline assessments.

• Administered pretest survey and MNCH tutorials to FHWs in participating PHC facilities (September–October 2017).

The combination of gaps in the eHealth literature and increasing interest from policy-makers and funders in researches focusing on practical issues create a favourable environment for this study to generate new knowledge. The study findings will provide a timely contribution to ongoing debate about effectiveness of eHealth approaches for improving quality and efficiency of health systems functions and client outcomes. In line with this, specific impacts of our study on policy and practice in Nigeria and internationally will include:

• Clarifying how using SatCom technology to scale up eHealth interventions contributes to health systems strengthening in Nigeria.

• Improving understanding of the effectiveness, acceptability and benefits of eHealth solutions for staff training and data management.

• Clarifying key contextual determinants of success of eHealth solutions in low-income and middle-income countries.

The authors also wish to acknowledge the contributions of staff at the Federal and the States Ministry of Health in Ondo State to the study design.

Footnote

Contributors BE, OO, BO and TO jointly conceived the study. BE, JNN, JH, TM, BO, OO, TO and TJ developed the study proposal. MA led the writing of this paper with contributions from BE, BO, GA, JT, KO, DA, AA, OD, RY, JH, OO, TO, TJ, AIS, TM and JNN. All authors read and approved the final version of the manuscript.

Funding This article presents independent research funded by the UK Space Agency International Partnership Programme. The sponsor and funders had no involvement in the study design; collection, management, analysis, and interpretation of data, writing of the report, and the decision to submit the report for publication. The study is funded by the UK Space Agency, Grant reference number IPPC1-30.

Disclaimer All views expressed in this publication are of the authors only.

Competing interests OO is Co-Founder and CEO of InStrat Global Health Solutions, the company that will implement the eHealth tools used in the outlined research programme. TO is Programme Director and TJ a development consultant for Inmarsat, the company providing satellite communication capability to deliver eHealth tools during the research programme. All other authors declare having no competing interests (BE, MA, BO, GA, JT, KO, DA, AA, OD, RY, JH, AIS, TM and JM).

Patient consent Not required.

Ethics approval Ethical approval for the study was granted by the University of Leeds School of Medicine Research Ethics Committee (MREC16-178), the Ondo State Government Ministry of Health (AD.4693 Vol. II/109), the Kano State Ministry of Health (MOH/Off/797/T1/350) and the Federal Capital Health Research Ethics Committee (FHREC/2017/01/42/12-05-17).

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Shaw T, McGregor D, Brunner M, et al. What is eHealth (6)? Development of a Conceptual Model for eHealth: Qualitative Study with Key Informants. J Med Internet Res 2017; 19: e324. doi:10.2196/jmir.8106

² Piette JD, Lun KC, Moura LA, et al. Impacts of e-health on the outcomes of care in low- and middle-income countries: where do we go from here? Bull World Health Organ 2012; 90: 365–72. doi:10.2471/BLT.11.099069

³ Eysenbach G. What is e-health? J Med Internet Res 2001; 3: e20. doi:10.2196/jmir.3.2.e20

⁴ Obasola OI, Mabawonku I, Lagunju I. A Review of e-Health Interventions for Maternal and Child Health in Sub-Sahara Africa. Matern Child Health J 2015; 19: 1813–24. doi:10.1007/s10995-015-1695-0

⁵ Ross J, Stevenson F, Lau R, et al. Exploring the challenges of implementing e-health: a protocol for an update of a systematic review of reviews. BMJ Open 2015; 5: e006773. doi:10.1136/bmjopen-2014-006773

⁶ de la Torre-Díez I, López-Coronado M, Vaca C, et al. Cost-utility and cost-effectiveness studies of telemedicine, electronic, and mobile health systems in the literature: a systematic review. Telemed J E Health 2015; 21: 81–5. doi:10.1089/tmj.2014.0053

⁷ Elbert NJ, van Os-Medendorp H, van Renselaar W, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res 2014; 16: e110. doi:10.2196/jmir.2790

⁸ Meier CA, Fitzgerald MC, Smith JM. eHealth: extending, enhancing, and evolving health care. Annu Rev Biomed Eng 2013; 15: 359–82. doi:10.1146/annurev-bioeng-071812-152350

⁹ Smith R, Menon J, Rajeev JG, et al. Potential for the use of mHealth in the management of cardiovascular disease in Kerala: a qualitative study. BMJ Open 2015; 5: e009367. doi:10.1136/bmjopen-2015-009367

¹⁰ Wamala DS, Augustine K. A meta-analysis of telemedicine success in Africa. J Pathol Inform 2013; 4: 6. doi:10.4103/2153-3539.112686

¹¹ Comm B, 2017. Nigeria - Mobile infrastructure, operators and broadband - statistics and analyses. Secondary Nigeria - mobile infrastructure, operators and broadband - statistics and analyses. https://www.budde.com.au/Research/Nigeria-Mobile-Infrastructure-Operators-and-Broadband-Statistics-and-Analyses

¹² GSMA. Digital inclusion and the role of mobile in Nigeria. London: GSMA, 2015.

¹³ PricewaterhouseCoopers. Cost Benefit Analysis of Satellite-Enhanced Telemedicine and eHealth Services in Sub-Saharan Africa. London: European Space Agency, 2008.

¹⁴ Srivastava S, Pant M, Abraham A, et al. The Technological Growth in eHealth Services. Comput Math Methods Med 2015; 2015: 1–18. doi:10.1155/2015/894171

¹⁵ Fullman N, Barber RM, Abajobir AA, et al. Measuring progress and projecting attainment on the basis of past trends of the health-related Sustainable Development Goals in 188 countries: an analysis from the Global Burden of Disease Study 2016. Lancet 2017; 390: 1423–59. doi:10.1016/S0140-6736(17)32336-X

¹⁶ O’Donovan J, Bersin A, O’Donovan C. The effectiveness of mobile health (mHealth) technologies to train healthcare professionals in developing countries: a review of the literature. BMJ Innov 2015; 1: 33–6. doi:10.1136/bmjinnov-2014-000013

¹⁷ Agarwal S, Perry HB, Long L-A, et al. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Tropical Medicine & International Health 2015; 20: 1003–14. doi:10.1111/tmi.12525

¹⁸ Agarwal S, Perry HB, Long LA, et al. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Trop Med Int Health 2015; 20: 1003–14. doi:10.1111/tmi.12525

¹⁹ Agarwal S, Lefevre AE, Labrique AB. A call to digital health practitioners: new guidelines can help improve the quality of digital health evidence. JMIR Mhealth Uhealth 2017; 5: e136. doi:10.2196/mhealth.6640

²⁰ Mirzoev T, Etiaba E, Ebenso B, et al. Study protocol: realist evaluation of effectiveness and sustainability of a community health workers programme in improving maternal and child health in Nigeria. Implement Sci 2016; 11: 1–11. doi:10.1186/s13012-016-0443-1

²¹ Hayes RJ, Moulton LH. Cluster randomised trials. Boca Raton: CRC Press, 2017.

²² Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess R, eds. Analyzing Qualitative Data. New York: Routledge, 1994.

²³ COMDIS-HSD, 2013. Research and development for effective health service delivery secondary research and development for effective health service delivery. http://comdis-hsd.dfid.gov.uk/about-us/

²⁴ COMDIS-HSD. How to’ guide on research & development: the embedded approach. Leeds, UK: COMDIS-Health Service Delivery, 2012.

²⁵ Ebenso B, Huque R, Azdi Z, et al. Protocol for a mixed-methods realist evaluation of a health service user feedback system in Bangladesh. BMJ Open 2017; 7: e017743. doi:10.1136/bmjopen-2017-017743

²⁶ Mirzoev TN, Omar MA, Green AT, et al. Research-policy partnerships - experiences of the Mental Health and Poverty Project in Ghana, South Africa, Uganda and Zambia. Health Res Policy Syst 2012; 10: 30. doi:10.1186/1478-4505-10-30

²⁷ Otu A, Ebenso B, Okuzu O, et al. Using a mHealth tutorial application to change knowledge and attitude of frontline health workers to Ebola virus disease in Nigeria: a before-and-after study. Hum Resour Health 2016; 14: 1–9. doi:10.1186/s12960-016-0100-4