Clinic-based evaluation study of the diagnostic accuracy of a dual rapid test for the screening of HIV and syphilis in pregnant women in Nigeria

Abstract

Background

Screening pregnant women for HIV and syphilis is recommended by WHO in order to reduce mother-to-child transmission. We evaluated the field performance, feasibility, and acceptability of a dual rapid diagnostic test (RDT) for HIV and syphilis test in antenatal clinic settings in Nigeria.

Methods and findings

Participants were recruited at 12 antenatal clinic sites in three states of Nigeria. All consenting individuals were tested according to the national HIV testing algorithm, as well as a dual RDT, the SD BIOLINE HIV/Syphilis Duo Test (Alere, USA), in the clinic. To determine sensitivity, specificity and concordance, whole blood samples were obtained for repeat RDT performance in the laboratory, as well as reference tests for HIV and syphilis. Dual test acceptability and operational characteristics were assessed among participants and clinic staff.

The prevalence of HIV among the 4,551 enrollees was 3.0% (138/4551) using the national clinic-based HIV testing algorithm. Positive and negative percent agreement of the HIV component of the dual RDT were 100.0% (95% CI 99.7–100.0) and 99.9% (95% CI 99.7–100.0) respectively, when compared with the national rapid testing algorithm. The prevalence of syphilis, using TPHA as the reference test, was low at 0.09% (4/4550). The sensitivity of the syphilis component of the dual RDT could not be calculated as no positive results were observed for patients that were positive for syphilis by TPHA. Each of the only four TPHA-positive specimens had RPR titers of 1:1 (neat), indicative of non-active syphilis. The specificity of the syphilis component of the dual RDT was 99.9% (95% CI 99.8–100.0).

The dual RDT received favorable feasibility ratings among antenatal care clinic staff. Acceptability among study participants was high with most women reporting preference for rapid dual HIV/syphilis testing.

Conclusions

The SD BIOLINE HIV/Syphilis Duo Test showed a high overall diagnostic accuracy for HIV and a high specificity for syphilis diagnosis in antenatal clinic settings. This study adds to a growing body of evidence that supports the clinic-based use of dual tests for HIV and syphilis among pregnant women.

About the Authors:

Ijaodola Olugbenga

Roles Conceptualization, Data curation, Investigation, Methodology, Project administration, Resources, Supervision, Visualization, Writing – review & editing

Affiliation: National AIDS and STIs Control Programme, Federal Ministry of Health Nigeria, FCT Abuja, Nigeria

Oyelade Taiwo

Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Supervision

Affiliation: WHO Regional Office for Africa, Brazzaville, Congo

ORCID logo http://orcid.org/0000-0002-3452-975X

Maura Laverty

Roles Conceptualization, Investigation, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing

Affiliation: Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland

Evelyn Ngige

Roles Data curation, Investigation, Project administration

Affiliation: National AIDS and STIs Control Programme, Federal Ministry of Health Nigeria, FCT Abuja, Nigeria

Chukwuma Anyaike

Roles Conceptualization, Investigation, Project administration

Affiliation: National AIDS and STIs Control Programme, Federal Ministry of Health Nigeria, FCT Abuja, Nigeria

Rasheed Bakare

Roles Investigation, Project administration

Affiliation: Department of Microbiology, University College Hospital Ibadan, Ibadan, Nigeria

Veronica Ogunleye

Roles Investigation, Project administration

Affiliation: Department of Microbiology, University College Hospital Ibadan, Ibadan, Nigeria

Brandy L. Peterson Maddox

Roles Data curation, Formal analysis, Software, Writing – review & editing

Affiliation: Division of STD Prevention, National Center for HIV, Viral Hepatitis, STD and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America

Daniel R. Newman

Roles Data curation, Formal analysis, Software, Writing – review & editing

Affiliation: Division of STD Prevention, National Center for HIV, Viral Hepatitis, STD and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America

ORCID logo http://orcid.org/0000-0002-9932-8940

Harriet D. Gliddon

Roles Data curation, Formal analysis, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliations Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland, London Centre for Nanotechnology, University College London, London, United Kingdom

ORCID logo http://orcid.org/0000-0002-9623-4252

Eugenia Ofondu

Roles Investigation, Project administration

Affiliation: Dermatology and Venereology Department, Federal Medical Center Owerri, Owerri, Nigeria

Stephen Nurse-Findlay

Roles Conceptualization, Writing – review & editing

Affiliation: Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland

ORCID logo http://orcid.org/0000-0002-9582-9254

Melanie M. Taylor

Roles Conceptualization, Funding acquisition, Methodology, Software, Supervision, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland, Division of STD Prevention, National Center for HIV, Viral Hepatitis, STD and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America

ORCID logo http://orcid.org/0000-0002-1786-6295

Introduction

The global burden of mother-to-child transmission of HIV and syphilis continues to disproportionately affect populations in low and middle-income countries and reflects access to quality antenatal care services that include HIV and syphilis testing and treatment. Global and regional strategies and initiatives have been launched for dual elimination of mother-to-child transmission (EMTCT) of HIV and syphilis [1] [2] [3] [4], eleven countries have been validated for elimination to date [1], and a number of tools have been developed available to accelerate this [5]. Early detection and timely intervention of pregnant women infected with HIV and/or syphilis are required services indicators for country validation of EMTCT [1].

HIV transmission from mother-to-child occurs in approximately 15–30% of untreated pregnancies [4]. In 2015, Nigeria had the greatest number of new HIV infections among children in the world—an estimated 41,000 [range 28,000–57,000]—roughly equivalent to the next eight highest incidence countries combined. There has been a limited decline in new pediatric HIV infections in Nigeria since 2009; 21% compared to the 60% average among the other Global Plan priority countries [6].

Untreated maternal syphilis results in adverse pregnancy outcomes due to congenital syphilis in over half of affected pregnancies and can lead to early fetal loss, premature birth, stillbirth, neonatal death, low birth weight, and infant complications from infection [7]. Syphilis diagnosis and treatment is one of the most cost-effective and feasible intervention in low-resource and low prevalence settings [8] but studies show that antenatal screening coverage is approximately 38% in sub-Saharan Africa [9]. In 2015, in Nigeria, only 13.7% of pregnant women nationally were screened for syphilis and of those infected (1.2%) only 61% were estimated to have received treatment [10]. With an annual birth rate of over seven million annually [11], the contribution of congenital syphilis to adverse birth outcomes in Nigeria may be high. The need for prompt diagnosis and appropriate treatment of all infected pregnant women cannot be over emphasized in the bid for the EMTCT of HIV and syphilis.

Presently in Nigeria, diagnosis of HIV infection and syphilis in pregnant women is primarily achieved using separate tests. This has been noted to be time consuming for both the client and healthcare providers and associated with significant loss to follow-up for treatment of syphilis-positive cases among antenatal clinic (ANC) attendees. While HIV testing coverage in ANCs is fairly high in Nigeria, syphilis testing coverage in pregnant women is very low [10] [12]. One of the causes for this is likely to be that HIV testing is free, while syphilis testing is not. In addition, when syphilis testing is done, same day results are not available and treatment is delayed. Further, pregnant women testing positive for syphilis frequently have to pay for the benzathine benzylpenicillin treatment and its administration as an intramuscular injection.

Nigeria is amongst the countries prioritized for dual elimination of mother to child transmission of HIV and congenital syphilis [12]. Dual rapid diagnostic tests (RDTs) with adequate levels of performance may improve the timely detection of and treatment of HIV and syphilis infections among pregnant women in developing countries, which will facilitate elimination of mother-to-child transmission of HIV and syphilis when employed in antenatal settings [12, 13]. Recently, dual RDTs, used at the point-of-care for simultaneously detecting antibodies to HIV and Treponema pallidum, the causative agent of syphilis (dual HIV & syphilis treponemal RDTs) using venous whole blood, serum/plasma, or finger-stick whole blood have been developed and are now commercially available [13] [14] [15]. To date, there are few data on the performance of these dual RDTs in ANC settings, although they have been evaluated in several laboratory-based and field-based studies and have shown encouraging sensitivities and specificities as compared with reference laboratory tests [15].

In 2014, Nigeria performed a laboratory-based performance study of three dual HIV/syphilis RDT kits [12]. Subsequent to the laboratory evaluation of commercially available dual RDTs for HIV and syphilis, a donation of the SD BIOLINE HIV/Syphilis Duo RDT by Alere (USA) was made to the National Ministry of Health through the Organization of African First Ladies against AIDS (OAFLA). Nigeria chose this opportunity to evaluate the performance of this WHO-prequalified test kit as well as its feasibility and acceptability from the perspective of both the client and the healthcare worker. Twelve ANCs in three states in Nigeria were identified as study sites for this evaluation.

The primary objective of this study was to determine the diagnostic accuracy performance of the SD BIOLINE HIV/Syphilis Duo Test (the ‘dual RDT’) for the screening of HIV and syphilis in pregnant women compared to that of (1) the national HIV testing algorithm and (2) a laboratory-based 4th generation HIV Enzyme Immunoassay (EIA) as two reference standards for HIV, and the T. pallidum Hemagglutination assay (TPHA) as the syphilis reference standard. The national HIV testing algorithm currently relies on HIV diagnosis using the DetermineTM HIV-1/2 Test (Alere), confirmed by Uni-Gold HIV Rapid Test (Trinity Biotech, Wicklow, Ireland). The 4th generation HIV EIA used was the Genscreen Ultra HIV Ag-Ab Assay (BIO-RAD Europe GmbH). Since it detects HIV antigens as well as antibodies, it is more sensitive than tests that detect HIV antibodies alone, as is the case for Determine HIV-1/2 Test and the dual RDT, particularly in the acute stages of infection.

The secondary objective was to assess the acceptability and operational characteristics of the dual test among pregnant women and antenatal care clinic staff.

Methods

Ethics statement

Study protocol and consent was approved by WHO Ethics Research Committee (3rd February 2017; protocol ID: A65909) and the National Health Research Ethics Committee of Nigeria (15th November 2016; assigned number: NHREC/01/01/2007) prior to study initiation. All participants provided written informed consent for the diagnostic accuracy study and the operational characteristics questionnaire, and these were recorded and stored by the study team.

Study site selection

A rapid assessment of facilities was conducted to assess and select suitable sites for this prospective study, and to identify study teams from each selected clinic facility. The selection criteria for the study sites included: (1) access to a sufficiently large population of pregnant women attending their first ANC visit in order to complete patient recruitment within 12 weeks of study initiation; (2) close proximity to the reference laboratory so that whole blood for reference tests could be processed and stored within eight hours of collection; (3) estimated prevalence of HIV and syphilis prevalence sufficiently high to achieve study parameters for performance analysis. Three reference laboratories and 12 enrollment sites (S1 Table) were selected in three states (Oyo, Imo and the Federal Capitol Territory (FCT)) in Nigeria. The sites chosen were healthcare facilities providing ANC services including HIV and syphilis testing.

The selection criteria for the reference laboratories selected included: (1) routine availability of reference testing for HIV and syphilis; (2) evidence of ongoing accreditation for laboratory quality assurance and management systems; (3) sufficiently trained laboratory and site staff with capacity to perform the study in accordance with the study protocol; (4) strong interest and commitment to research. Study teams consisted of nurses, physicians, laboratory scientists, phlebotomists, and record officers. Supervisory level staff included the state and site coordinator, data managers and reference laboratory coordinators.

Study participants: Inclusion and exclusion criteria

Eligible participants included pregnant women presenting for their first ANC visit at one of the 12 study sites. Women were excluded if they were less than 18 years of age or could not provide informed consent.

Pre-enrollment study activities and staff training

Study training was conducted over two seven-hour training sessions that took place over two consecutive days in each of the three States, FCT, Oyo and Imo, prior to study initiation. This included training on informed consent, data collection forms, data entry, and required study sample collection. S1 Fig shows an example training agenda. All study staff attended these trainings (S2 Table).

Hands-on training was provided by the dual RDT manufacturing company to all study staff who would be performing the tests in the field and in the laboratory setting. At least five staff members from each facility were trained in the use of the dual RDT. These included the study coordinators, ANC nurses, the laboratory phlebotomists, and the record officers from each site. Based on a simple training module, a pictorial training chart, appropriate for the local cultural context, was developed and used to train on the method to perform the tests according to manufacturer guidelines and to records results accurately.

Study materials

The 10,100 SD BIOLINE HIV/Syphilis Duo Tests (Alere, USA) used for this study were donated by the manufacturer to the Nigerian government on June 2, 2016 and contained an expiration date of May 13, 2017. No other financial support from manufacturers was received. Other study items and materials were obtained through WHO funding. This included consumables, data collection forms, and benzathine benzylpenicillin.

Sample size calculations for operational performance assessment

The sample size calculation was based on the laboratory performance of the dual RDT compared to the reference standard and the estimated HIV and syphilis prevalence in Nigeria. A disease seroprevalence of 5% for both HIV and syphilis in pregnant women was assumed based on recent surveillance data (2015), an assumed higher risk of HIV and/or syphilis in pregnant women, and the low uptake of HIV and syphilis testing in ANCs in Nigeria. Using an estimated test sensitivity of 98%, an alpha value of 0.05 and a beta value of 80% (power), a sample size of 3,800 was estimated. Based on the impreciseness of the available prevalence rates, particularly for syphilis, the sample size was increased to 4,500 women.

Participant enrollment

Fig 1 describes the step-by-step enrollment, including informed consent and testing procedures used during the study to ensure uniformity across selected sites and consistency with the protocol. Roles and responsibilities of the study implementation team are described in S1 Table.

[Figure omitted. See PDF.]

Fig 1. Recruitment and enrollment process of participants in ANC clinics.

https://doi.org/10.1371/journal.pone.0198698.g001

Use of the dual RDTs and collection of specimens

All consenting ANC attendees were enrolled by a trained study nurse. The fingerprick blood sample obtained was used to perform the SD BIOLINE HIV/Syphilis Duo Test according to the manufacturer’s instructions, as well as the national HIV testing algorithm with DetermineTM HIV-1/2 Test, confirmed by Uni-Gold HIV Rapid Test. After 15–20 minutes, the nurse informed participants of the result of the DetermineTM HIV-1/2 Test and the syphilis result of the dual RDT and recorded all results on the socio-demographic questionnaire. In addition, 5ml of venipuncture whole blood was collected into an EDTA collection tube which was stored and then transported to the reference laboratory, in a temperature-controlled container (15°C) within eight hours of specimen collection.

Upon arrival at the reference laboratory, a repeat dual RDT was performed on the whole blood specimen, by the laboratory staff, who were blinded to prior test results and clinic information of the participants (Fig 2). Study samples were then processed and stored as per manufacturer instructions for each of the tests performed in the laboratory. Two (2ml) aliquots of plasma were pipetted into cryovials and stored in a -70°C freezer. For syphilis reference testing, fresh whole blood specimens were tested with the TPHA. Those specimens that tested positive by TPHA were then tested using a quantitative RPR and the titer was recorded. The time interval between conducting the TPHA and the RPR, if required, on refrigerated samples was a maximum of one week. HIV reference tests were performed using stored samples at the end of the study using the Genscreen Ultra HIV Ag-Ab Assay, a 4th generation HIV test. Frozen cryovials were shipped on dry ice from Imo and FCT states to the reference lab in Oyo state for performance of the HIV reference test. The time interval between the field-performed HIV tests and the batch testing of the Genscreen Ultra HIV Ag-Ab Assay was up to four months as serum was stored (-70°C) and batch tested at the end of the study. However, for a given participant, all reference samples in the study were taken at the same time as the field-performed dual RDT. All test results were recorded in a laboratory log.

[Figure omitted. See PDF.]

Fig 2. Workflow used for laboratory processing of specimens.

https://doi.org/10.1371/journal.pone.0198698.g002

Data collection for feasibility and acceptability analysis

As part of the enrollment process, the consenting women completed the 10-item Socio-Demographic Participant Questionnaire (S2 Fig) to determine acceptability of the investigational dual RDT. Acceptability among participants was determined by asking the women if they would be willing to wait for their test results and if so, how long they would be willing to wait for these results (i.e., 30 minutes, one hour or two hours). Participants were also asked if they preferred a single test or dual test for HIV and syphilis.

Assessment of the feasibility of the dual RDT was conducted among ANC staff members. A brief six-item Dual Test Operational Characteristics Questionnaire (S3 Fig) was administered to ANC clinic staff (i.e. providers, nurses, laboratory technicians) from each field evaluation site, who performed and read the test, to determine operational characteristics by rating six test attributes (e.g. clarity of instructions, ease of use, and readability of test results). ANC clinic staff completed the operational characteristics questionnaire at the end of participant enrollment. Consent to complete the operational characteristics questionnaire was obtained prior to providing the survey to ANC staff. Study site coordinators were responsible for collecting and managing the participant and ANC staff questionnaires.

Feasibility and acceptability data analysis

CDC DSTDP staff led the data analysis and did not have contact with human subjects for this project component. Microsoft Excel was used for initial data management and quantitative data analysis was conducted using SAS (Version 9.4). Quantitative data were principally analyzed using descriptive statistics. Data from staff and participant questionnaires were reviewed for dual test preferences and other pertinent test acceptability information. Proportions were used to categorize and summarize the baseline characteristics.

Analysis of acceptability of the dual RDT was performed using the proportion of pregnant women who preferred dual RDT test as well as the proportion willing to wait for test results and for how long. To assess the operational feasibility of the dual RDT, ANC clinic staff rated test attributes (i.e., minimum operational characteristic) on a scale of 0–2 (2 attributes) or 0–3 (4 attributes) and we calculated the mean score for all staff who responded to the questionnaire for each attribute. Attributes scaled 0–2 with a score ≥ 1.50 and scaled 0–3 with a score ≥ 2.25 were coded acceptable. An overall feasibility score was calculated as the total of the mean attribute scores.

Diagnostic test accuracy statistical analysis

The sensitivity and specificity of the dual RDT was assessed using two separate reference standards for HIV; (1) the national rapid testing algorithm with DetermineTM HIV-1/2 Test (Alere) positivity confirmed by a positive Uni-Gold HIV Rapid Test (Trinity Biotech) and (2) the laboratory-based HIV 4th generation Ag/Ab Enzyme Immunoassay (EIA) (Genscreen ULTRA HIV Ag-Ab Assay). This allowed a comparison between the dual RDT and the current HIV testing standard in ANC settings in the field, and also a gold standard laboratory-based assay. The accuracy of the syphilis component was assessed using the TPHA (Human Gesellschaft fur Biochemica und Diagnostica mbH, Wiesbaden, Germany) as a laboratory reference standard for antibodies to T. pallidum. A rapid plasma reagin (RPR) assay (Syphilis RPR test, Human Gesellschaft fur Biochemica und Diagnostica mbH, Wiesbaden, Germany) was conducted for samples that gave a positive TPHA result in order to identify acute infection (RPR≥ 1:4).

The sensitivity, specificity and kappa statistics [16] were calculated using the EpiBasix package [17] in R statistics (version 3.4.0) [18]. 95% confidence intervals were calculated using the exact binomial method.

Results

Study population

Table 1 lists the characteristics of the 4,551 study participants recruited across the 12 selected study sites. The median age of the women recruited to the study was 30 years (Interquartile range (IQR): 27–33). The median gestational age was 20 weeks (IQR: 16–28) and the median time (minutes) taken to reach the clinic from their homes was 30 minutes (IQR: 30–50).

[Figure omitted. See PDF.]

Table 1. Characteristics of study participants, including their age, gestation length and the time required to reach the ANC clinic from home.

https://doi.org/10.1371/journal.pone.0198698.t001

Specimen collection and reporting of results

All participants had a dual RDT, Determine HIV-1/2 Test (Alere) (as well as a Uni-Gold™ HIV Rapid Test (Trinity Biotech) for confirmation of a positive result, and a HIV 1/2 STAT-PAK® Assay (Chembio) test upon a negative Uni-Gold™ HIV Rapid Test result) at the ANC clinic. Participants received HIV results from the national HIV testing algorithm and the syphilis component of the RDT. A completed STARD checklist and all diagnostic test accuracy results can be found in the supplementary information (S1 File and S2 File respectively).

Serum samples used for laboratory testing with the dual RDT and reference tests for antibodies to HIV and syphilis were collected from 4,550 participants. One participant experienced failed attempts at blood draw and was therefore unable to be tested using the HIV EIA reference testing. However, rapid test results for the dual RDT and DetermineTM HIV-1/2 Test/Uni-Gold™ HIV Rapid Test were available from the ANC setting. Therefore, this individual could only be included in the analysis comparing the accuracy of the HIV component of the dual RDT to DetermineTM HIV-1/2 Test/Uni-Gold™ HIV Rapid Test. There were no other missing results, and no indeterminate or inconclusive results were given by any of the RDTs or laboratory assays used in the study.

Four participants tested positive for syphilis using the dual RDT, but negative when the TPHA was performed in the laboratory. For two of these participants, the dual RDT was repeated on the advice of senior clinical team during the same clinic visit with subsequent negative result for syphilis. Therefore, these two participants were not treated for syphilis. The false positive results were probably caused by incorrect interpretation of the dual RDT. Two participants received benzathine benzylpenicillin due to a positive syphilis dual RDT result, but subsequently had a negative TPHA result. This minor overtreatment is the only adverse effect of the study that the authors are aware of.

Prevalence of HIV and syphilis

Prevalence of HIV among study participants was 3.0% (138/4551) using the national clinic-based point-of-care HIV testing algorithm. Prevalence of TPHA positivity for syphilis was low at 0.09% (4/4550). Of the four TPHA positive results, all were positive by RPR only at a titer of 1:1, likely indicating previously treated infection.

HIV clinical sensitivity and specificity of the dual RDT in the field

HIV: Sensitivity and specificity of the dual RDT (field-performed) versus 4th generation HIV Ag/Ab laboratory reference (Table 2).

Compared to the laboratory-based 4th generation HIV Ag/Ab reference test, the performance of the HIV component of the dual RDT had a sensitivity of 85.8% (95% CI 79.1–90.6) and a specificity of 99.5% (95% CI 99.3–99.7) with a Kappa coefficient of 0.847 (95% CI 0.802–0.893).

[Figure omitted. See PDF.]

Table 2. Summary of diagnostic accuracy results.

https://doi.org/10.1371/journal.pone.0198698.t002

HIV: Nigeria national HIV testing algorithm (field-based DetermineTM HIV-1/2 Test with Uni-Gold™ HIV Rapid Test confirmation) versus 4th generation HIV Ag/Ab laboratory reference (Table 2).

Compared to the laboratory-based 4th generation HIV Ag/Ab reference test, the performance of the Nigeria national HIV testing algorithm included a sensitivity of 81.6% (95% CI 74.4–87.1) and specificity of 99.5% (95% CI 99.3–99.7) with Kappa coefficient of 0.822 (95% CI 0.772–0.871).

HIV: Dual RDT (field-performed) versus Nigeria national HIV testing algorithm (field-based DetermineTM HIV-1/2 Test with Uni-Gold™ HIV Rapid Test confirmation) (Table 2).

Although Nigeria’s national HIV testing algorithm is not a standard reference test for HIV diagnosis, it is nevertheless a useful comparison for the dual RDT. Where a non-standard reference test is used, sensitivity and specificty should be substituted for positive percent agreement and negative percent agreement, respectively. The positive percent agreement of the HIV component of the dual RDT was 100.0% (95% CI 99.7–100.0) compared with the national testing algorithm (clinic-based DetermineTM HIV-1/2 Test /Uni-Gold™ HIV Rapid Test) and negative percent agreement was 99.9% (95% CI 99.7–100.0) with a Kappa co-efficient of 0.978 (95% CI 0.960–0.996).

Syphilis clinical sensitivity and specificity of the SD BIOLINE HIV/Syphilis Duo test in the field

Comparison of the syphilis component of the dual RDT with laboratory-based TPHA (Table 2).

The sensitivity of the syphilis component of the dual RDT could not be calculated because there were no positive results for both the TPHA and the dual RDT. Four specimens out of the 4,550 tested were TPHA positive, but these were not identified as positive using the dual RDT. Each of these four specimens were positive by non-treponemal RPR testing but were reported as having a titer of 1:1 (neat), indicative of non-active syphilis.

The specificity of syphilis diagnosis using the dual RDT was 99.9% (95% CI 99.8–100.0), using TPHA as the laboratory reference standard for syphilis infection. There were four false positives, where a positive syphilis result was recorded for the dual RDT, but the TPHA test was subsequently found to be negative. Two of these occurred at the State Hospital Adeoyo (Oyo State) and were the only positive syphilis results seen at this site using the dual RDT. These two positive results occurred on the same day and within the first two weeks of the study, and they coincided on both occasions with a positive HIV result. It is possible that these two tests were mistakenly interpreted as positive for both HIV and syphilis rather than positive for HIV alone. Same day repeat testing with the RDT yielded negative syphilis results for these two participants.

Concordance of the dual RDT when conducted in laboratory and field settings (Table 3)

A high concordance between laboratory and field settings was observed for HIV diagnosis, with a kappa value of 0.967 (CI95% 0.946–0.989). However, the concordance for syphilis was lower, with a kappa value of 0.666 (CI95% 0.358–0.974).

[Figure omitted. See PDF.]

Table 3. Concordance for HIV and syphilis diagnosis by the SD BIOLINE HIV/Syphilis Duo test when conducted in laboratory and field settings.

https://doi.org/10.1371/journal.pone.0198698.t003

Dual RDT acceptability among participants

Acceptability data were obtained from the 4,551 pregnant women enrolled in the study. As displayed in Table 4, the level of acceptability of the dual RDT was relatively the same among all sites and pregnant women overwhelmingly reported a preference for the dual HIV/syphilis RDT (99.9%) compared to single tests for HIV and syphilis (0.1%). Only five women preferred a single test (0.1%) and one women stated the reason was because she wanted, ‘to be sure of different results’. Nearly all women were willing to wait at the clinic for their test results (99.9%). About three-quarters (73.8%) were willing to wait up to 30 minutes.

[Figure omitted. See PDF.]

Table 4. Acceptability of dual RDT among pregnant women attending antenatal care in Nigeria by baseline characteristics.

https://doi.org/10.1371/journal.pone.0198698.t004

Dual RDT acceptability and operational characteristics among ANC staff

Forty-eight ANC clinic staff completed surveys on the dual RDT’s minimum operational characteristics. The ANC staff characteristics are displayed in S3 Table. Completed surveys among clinic staff was relatively proportional across the three states, with the majority of responses from site nurses (50.0%) who had experience directly using the test as part of the study.

Regarding operational feasibility, the ANC clinic staff rated the dual RDT with an overall feasibility score of 12.3 out of a maximum obtainable score of 16 (Table 5). Clinic staff in Oyo State rated feasibility of the test higher than clinic staff in FCT and Imo State (13.1, 12.5, and 11.1, respectively). Overall, ANC staff provided an acceptable rating for 4 of 6 operational attributes, with the exception of ‘hands-on time’ (scale: 0–2, score: 1.35) and ‘training time required’ (scale: 0–3, score: 1.95). All results for the acceptability and operational characteristics of the dual RDT are listed in the supplementary information (S3 File).

[Figure omitted. See PDF.]

Table 5. Operational feasibility of dual HIV-syphilis rapid diagnostic test among ANC clinic staff by site.

https://doi.org/10.1371/journal.pone.0198698.t005

Discussion

This study is one of the few to assess field performance characteristics and acceptability of a novel, dual RDT for HIV and syphilis. Our findings are in line with previous studies, demonstrating high accuracy for diagnosis of HIV. Low prevalence of syphilis limited the ability to fully assess performance of this component of the test. The findings from the assessment of the acceptability and minimum operational characteristics of a dual HIV/syphilis RDT reflect similar findings in other studies assessing dual RDT acceptability and feasibility. The strength of this assessment is the ability to document test performance as well as perceptions of both pregnant women, the population likely to benefit from dual RDT use, and ANC staff with experience using the dual RDT in the clinic setting.

The positive percent agreement of the HIV component of the dual RDT compared to the national HIV testing algorithm (DetermineTM HIV-1/2 Test, +/- Uni-Gold™ HIV Rapid Test & HIV 1/2 STAT-PAK® Assay) was 100.0%; specificity was likewise high (99.9%). The sensitivities of both the dual RDT and the DetermineTM HIV-1/2 Test (3rd generation HIV EIA assays) were lower when compared to the 4th generation Ag-Ab test, a result to be expected due to the different generation of HIV EIA assays. We were unable to elucidate whether the additional detection of p24 by the 4th generation reference test explains the observed low sensitivity of the rapid tests, because the Genscreen ULTRA HIV Ag-Ab Assay test does not provide separate results for HIV-1/2 antigens and HIV-1/2 antibodies. Although the sensitivity of the dual RDT was higher for HIV diagnosis than the national HIV testing algorithm, this difference was not statistically significant, as the confidence intervals for each of the sensitivities overlapped. The prevalence of syphilis was too low to calculate the sensitivity of the syphilis component of the dual RDT but specificity was high (99.9%). Of the four participants that tested positive via the syphilis TPHA laboratory reference test (all four RDT syphilis-negative) all had low-titer RPR values of 1:1 likely reflecting old treated infection. Previous studies have shown a reduced sensitivity of dual RDTs for syphilis cases where the RPR titer is low [19, 20]. This has significant implications for training with the use of dual RDTs, since positive test responses for HIV and syphilis have been shown to appear with differing levels of band color intensity, with bands of darker intensity observed for HIV than syphilis [21, 22]. The concordance of the laboratory performance versus field-performance of the HIV component of the RDT was high (Kappa 0.967) however, for syphilis the concordance of the RDT was lower (Kappa 0.666) likely due to training issues regarding the syphilis test result interpretation and limited experience with use of this RDT.

Patient acceptability of the dual RDT was positive. Almost all women preferred the dual RDT and the majority were willing to wait up to 30 minutes for their test results. Nearly half of the women noted experiencing travel times to the clinic of greater than 30 minutes. Their willingness to wait up to 30 minutes for results of the dual RDT may be a more favorable choice than having to return to the clinic at a later time for syphilis results and treatment. The dual RDT received favorable feasibility ratings among antenatal care clinic staff with experience using the test in a clinic-based setting. Assessing the acceptability and minimum operational characteristics of this dual HIV/syphilis RDT provides timely and useful information beyond test performance for the Nigerian Ministry of Health (MoH) in consideration of use and scale-up use of the test across the country.

There are limitations that should be considered regarding the interpretation of these data. First, this was a convenience sample of pregnant women attending their first ANC visit limiting representativeness of pregnant women in Nigeria overall. Participant enrollment was limited to a specific time period reflecting the expiration dates of the donated test kits. The sensitivity of the syphilis component of the dual RDT was a key expected result from the study. However, a low prevalence of syphilis in this large study size of 4,551 pregnant women was unanticipated and prevented this calculation. This low syphilis prevalence is not representative of recent syphilis prevalence survey estimates in Nigeria. External quality assurance of HIV and syphilis testing was not performed through an outside reference lab. However, each of the reference laboratories were nationally accredited. Finally, use of the 4th generation HIV Ag/Ag test (Genscreen ULTRA HIV Ag-Ab Assay) as the laboratory reference standard for HIV testing did not reflect a true comparison with the HIV EIA component of the dual RDT. The ability of the 4th generation HIV to detect acute HIV infection prior to antibody seroconversion was reflected in the calculated sensitivities for the dual RDT test as well as the national rapid HIV testing algorithm which uses DetermineTM HIV-1/2 Test as the initial screening test.

The results from this assessment are being used by the Nigerian MoH to facilitate initial decision-making for future use and potential scale-up of the dual test in ANC settings. As the Nigerian MoH reviews these favorable outcomes, the next step potentially is how this information will be used to inform national policies, HIV and syphilis testing algorithms, and clinic practices of the sites included in the evaluation. An in-depth qualitative assessment may help the MoH identify ways to address barriers and implement strategies to improve acceptability and feasibility of dual HIV/syphilis testing provision and develop a plan of action to address key barriers.

Ministries of Health, outside of Nigeria, considering a similar implementation may be interested in the evaluation findings and this approach to validate test performance and assess acceptability, feasibility and dual testing barriers and facilitators within their own countries. This study adds to a growing body of evidence that supports the clinic-based use of dual tests for HIV and syphilis among pregnant women and can be used as further justification of their use in antenatal settings.

Supporting information

[Figure omitted. See PDF.]

S1 Table. Study enrollment sites consisted of 12 ANCs in three states in Nigeria.

https://doi.org/10.1371/journal.pone.0198698.s001

(PDF)

S2 Table. Study personnel and site roles and responsibilities.

https://doi.org/10.1371/journal.pone.0198698.s002

(PDF)

S3 Table. Characteristics of clinic staff assessing dual test operational characteristics.

https://doi.org/10.1371/journal.pone.0198698.s003

(PDF)

S1 Fig. Example agenda for pilot training.

https://doi.org/10.1371/journal.pone.0198698.s004

(PNG)

S2 Fig. Socio-demographic participant questionnaire used in the study.

https://doi.org/10.1371/journal.pone.0198698.s005

(PNG)

S3 Fig. Dual test operational characteristics questionnaire used in the study.

https://doi.org/10.1371/journal.pone.0198698.s006

(PNG)

S1 File. STARD checklist.

https://doi.org/10.1371/journal.pone.0198698.s007

(DOCX)

S2 File. Diagnostic test accuracy data.

https://doi.org/10.1371/journal.pone.0198698.s008

(XLSX)

S3 File. Dual RDT acceptability and operational characteristics data.

https://doi.org/10.1371/journal.pone.0198698.s009

(XLSX)

Acknowledgments

We thank the Honourable Minister FMOH, Dr Sunday Aboje (National Coordinator HIV FMOH), Dr Benson Udu (AIDS Coordinator FCT), Ms Teclaire Ndomg (Reference Laboratory FCT), Mr Amadi Ikechukwu (Reference Laboratory Owerri) and the Study Teams in Ibadan, Owerri and FCT Study Centers.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the US Centers for Disease Control and Prevention, World Health Organization, or the Federal Ministry of Health (Nigeria). The authors wish to thank all of the participants in this study. They also acknowledge the study staff at participating field evaluation sites.

Citation: Olugbenga I, Taiwo O, Laverty M, Ngige E, Anyaike C, Bakare R, et al. (2018) Clinic-based evaluation study of the diagnostic accuracy of a dual rapid test for the screening of HIV and syphilis in pregnant women in Nigeria. PLoS ONE 13(7): e0198698. https://doi.org/10.1371/journal.pone.0198698

References

1. WHO. Global Guidance on Criteria and Processes for Validation: Elimination of Mother-to-Child Transmission of HIV and Syphilis. Geneva: WHO. 2014.

2. PAHO. Elimination of Mother-to-Child Transmission of HIV and Syphilis in the Americas: 2015 update. wwwirispahoorg/xmlui/bitstream/handle/123456789/18372/9789275118702_engpdf?sequence=3&isAllowed=y (accessed December 27, 2016). 2015.

3. WHO. Global health sector strategy on Sexually Transmitted Infections, 2016–2021. Geneva: WHO. 2016.

4. WHO. Global health sector strategy on HIV, 2016–2021. Geneva: WHO. 2016.

5. Taylor M, Gliddon H, Nurse-Findlay S, Laverty M, Broutet N, Pyne-Mercier L, et al. Revisiting strategies to eliminate mother-to-child transmission of syphilis. Lancet Glob Health. 2018;6(1):e26–e8. Epub 2017/12/16. pmid:29241606.

6. UNAIDS. Prevention Gap Report. http://wwwunaidsorg/sites/default/files/media_asset/2016-prevention-gap-report_enpdf. 2016.

7. Gomez GB, Kamb ML, Newman LM, Mark J, Broutet N, Hawkes SJ. Untreated maternal syphilis and adverse outcomes of pregnancy: a systematic review and meta-analysis. Bull World Health Organ. 2013;91(3):217–26. pmid:23476094; PubMed Central PMCID: PMCPMC3590617.

8. Kahn JG, Jiwani A, Gomez GB, Hawkes SJ, Chesson HW, Broutet N, et al. The cost and cost-effectiveness of scaling up screening and treatment of syphilis in pregnancy: a model. PLoS One. 2014;9(1):e87510. pmid:24489931; PubMed Central PMCID: PMCPMC3906198.

9. WHO. Rreport on global sexually transmitted infection surveillance 2015. Geneva: World Health Organisation. 2015.

10. UNAIDS. Global AIDS Monitoring 2017. Geneva: UNAIDS http://wwwunaidsorg/sites/default/files/media_asset/2017-Global-AIDS-Monitoring_enpdf (accessed 27 June 2017). 2016.

11. UNICEF. Maternal and Newborn Health Disparities: Nigeria. https://datauniceforg/wp-content/uploads/country_profiles/Nigeria/country%20profile_NGApdf. 2016.

12. Yin YP, Ngige E, Anyaike C, Ijaodola G, Oyelade TA, Vaz RG, et al. Laboratory evaluation of three dual rapid diagnostic tests for HIV and syphilis in China and Nigeria. Int J Gynaecol Obstet. 2015;130 Suppl 1:S22–6. pmid:25975869.

13. WHO. WHO Information Note on the Use of Dual HIV/Syphilis Rapid Diagnostic Tests (RDT). Geneva: WHO. 2017.

14. WHO. World Health Organization Prequalification of In Vitro Diagnostics Programme, Public Report. Available: wwwwhoint/diagnostics_laboratory/evaluations/151028_final_report_0179-012-00_sd_bioline_hiv_syphilis2pdf?ua=1. 2015.

15. Gliddon HD, Peeling RW, Kamb ML, Toskin I, Wi TE, Taylor MM. A systematic review and meta-analysis of studies evaluating the performance and operational characteristics of dual point-of-care tests for HIV and syphilis. Sex Transm Infect. 2017. Epub 2017/07/28. pmid:28747410.

16. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. Epub 1977/03/01. pmid:843571.

17. Rotondi MA. Elementary Epidemiological Functions for Epidemiology and Biostatistics 2009.

18. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2016.

19. Hess KL, Fisher DG, Reynolds GL. Sensitivity and specificity of point-of-care rapid combination syphilis-HIV-HCV tests. PLoS One. 2014;9(11):e112190. pmid:25375138; PubMed Central PMCID: PMCPMC4223032.

20. Black V, Williams BG, Maseko V, Radebe F, Rees HV, Lewis DA. Field evaluation of Standard Diagnostics' Bioline HIV/Syphilis Duo test among female sex workers in Johannesburg, South Africa. Sex Transm Infect. 2016. pmid:27154184.

21. Bristow CC, Leon SR, Huang E, Brown BJ, Ramos LB, Vargas SK, et al. Field evaluation of a dual rapid diagnostic test for HIV infection and syphilis in Lima, Peru. Sex Transm Infect. 2016;92(3):182–5. pmid:26670914; PubMed Central PMCID: PMCPMC4875763.

22. Bristow CC, Severe L, Pape JW, Javanbakht M, Lee SJ, Comulada WS, et al. Dual rapid lateral flow immunoassay fingerstick wholeblood testing for syphilis and HIV infections is acceptable and accurate, Port-au-Prince, Haiti. BMC Infect Dis. 2016;16:302. pmid:27316352; PubMed Central PMCID: PMCPMC4912739.