Diagnostic accuracy of the Panbio COVID-19 antigen rapid test device for SARS-CoV-2 detection in Kenya, 2021: A field evaluation

Abstract

Background

Accurate and timely diagnosis is essential in limiting the spread of SARS-CoV-2 infection. The reference standard, rRT-PCR, requires specialized laboratories, costly reagents, and a long turnaround time. Antigen RDTs provide a feasible alternative to rRT-PCR since they are quick, relatively inexpensive, and do not require a laboratory. The WHO requires that Ag RDTs have a sensitivity ≥80% and specificity ≥97%.

Methods

This evaluation was conducted at 11 health facilities in Kenya between March and July 2021. We enrolled persons of any age with respiratory symptoms and asymptomatic contacts of confirmed COVID-19 cases. We collected demographic and clinical information and two nasopharyngeal specimens from each participant for Ag RDT testing and rRT-PCR. We calculated the diagnostic performance of the Panbio^™ Ag RDT against the US Centers for Disease Control and Prevention’s (CDC) rRT-PCR test.

Results

We evaluated the Ag RDT in 2,245 individuals where 551 (24.5%, 95% CI: 22.8–26.3%) tested positive by rRT-PCR. Overall sensitivity of the Ag RDT was 46.6% (95% CI: 42.4–50.9%), specificity 98.5% (95% CI: 97.8–99.0%), PPV 90.8% (95% CI: 86.8–93.9%) and NPV 85.0% (95% CI: 83.4–86.6%). Among symptomatic individuals, sensitivity was 60.6% (95% CI: 54.3–66.7%) and specificity was 98.1% (95% CI: 96.7–99.0%). Among asymptomatic individuals, sensitivity was 34.7% (95% CI 29.3–40.4%) and specificity was 98.7% (95% CI: 97.8–99.3%). In persons with onset of symptoms <5 days (594/876, 67.8%), sensitivity was 67.1% (95% CI: 59.2–74.3%), and 53.3% (95% CI: 40.0–66.3%) among those with onset of symptoms >7 days (157/876, 17.9%). The highest sensitivity was 87.0% (95% CI: 80.9–91.8%) in symptomatic individuals with cycle threshold (Ct) values ≤30.

Conclusion

The overall sensitivity and NPV of the Panbio^™ Ag RDT were much lower than expected. The specificity of the Ag RDT was high and satisfactory; therefore, a positive result may not require confirmation by rRT-PCR. The kit may be useful as a rapid screening tool only for symptomatic patients in high-risk settings with limited access to rRT-PCR. A negative result should be interpreted based on clinical and epidemiological information and may require retesting by rRT-PCR.

Introduction

The COVID-19 pandemic has had major negative socioeconomic effects [1]. Prompt diagnosis of SARS-CoV-2 infection is critical in controlling the pandemic [2]. Real-time reverse transcription-polymerase chain reaction (rRT-PCR) remains the reference standard for the diagnosis of SARS-CoV-2 infection [3]. However, rRT-PCR has limitations, including the need for specialized laboratories, equipment, and staff, longer turnaround time (TAT) for clinical decision making and prevention measures, high cost, and erratic supply of reagents to conduct SARS-CoV-2 rRT-PCR due to increased demand [3–6]. These limitations have increased the demand for more rapid, cheaper, and easy to perform testing methods [7].

Prompt identification of infected individuals is key in initiation of proper management of cases and the implementation of interventions to limit transmission [2]. Antigen rapid diagnostic tests (Ag RDTs) increase patients’ access to diagnosis of SARS-CoV-2 infection as they can be used as point of care tests thereby providing results more prompty. The World Health Organization (WHO) recommends Ag RDTs with a minimum sensitivity and specificity ≥80% and ≥97%, respectively where rRT-PCR is unavailable or is associated with long TAT [8].

The performance of Ag RDTs depends on the presence of symptoms, duration since onset, viral load and the disease prevalence [9]. At the time of study conception, Panbio^™ Ag RDT was one of two rapid tests approved for SARS-CoV-2 testing in our setup and to the best of our knowledge had not been evaluated. We evaluated the performance of the Panbio^™ Ag RDT compared to the reference standard rRT-PCR. The Panbio^™ Ag RDT is authorized in Kenya and received an emergency use license (EUL) from WHO [10]. We also assessed the operational characteristics of the Panbio^™ Ag RDT in order to identify any user challenges the in the field setting.

Methods

We conducted a prospective cross-sectional evaluation of the Panbio^™ Ag RDT in 11 sites in Kenya between March and July 2021. The sites were located in Nairobi County (Nairobi remand Prison, Langata Women’s Prison, and Tabitha level 2 Clinic), Kiambu County (Kihara Sub-county Hospital), Nakuru County (Nakuru Prison), Nyeri County (Nyeri level 5 Hospital and Nyeri Prison), Kisumu County (Kisumu Prison-Kodiaga), Mombasa County (Coast General Teaching and Referral Hospital) and Siaya County (Siaya County Referral Hospital and St. Elizabeth Mission Hospital). The protocol was reviewed and approved by the Kenya Medical Research Institute’s (KEMRI) scientific ethical review unit (SERU). The US Centers for Disease Control and Prevention (CDC) institutional review board provided non-research determination approval. Further administrative approvals were obtained from the Ministry of Health Kenya, the National Commission for Science, Technology, and Innovation (NACOSTI) Kenya, the Kenya Prisons Service, and the county governments of Nakuru, Nyeri, Mombasa, Kiambu, and Siaya. Written informed consent was obtained for all enrolled participants. Written parental informed consent and assent were obtained for participants aged <18 years. Individuals who declined to participate, including prisoners were offered routine COVID-19 services as per national guidelines.

Clinical case definition for enrollment

We enrolled persons of any age who presented to our study sites with respiratory symptoms (symptomatic participants) that met any of three case definitions, including acute respiratory infection (ARI,) defined as cough or difficulty in breathing or sore throat or coryza, with the onset of symptoms <2 weeks; severe acute respiratory infection (SARI) defined as fever or temperature ≥38°C and cough, with onset of symptoms <10 days and requiring hospitalization; or influenza-like illness (ILI) defined as temperature ≥38°C and cough, with onset of symptoms <10 days. Body temperature measurement was done using infrared thermometers placed 3 to 5 centimeters from the temple. We also enrolled asymptomatic close contacts of individuals with confirmed SARS-CoV-2 infection exposed between two days before and 14 days after symptoms onset or confirmation of infection. Individuals exposed to persons with confirmed SARS-CoV-2 infection in the following setups were considered close contacts in the following scenarios: working in close proximity or sharing the same environment; face-to-face contact within 1 meter for more than 15 minutes; traveling together in any kind of conveyance; living in the same household; healthcare-associated exposure, including providing direct care for COVID-19 patients, visiting patients or staying in the same close environment.

Data and specimen collection

Individuals who met the case definition and consented were enrolled and assigned unique identifiers. We used pretested questionnaires on Android^® smartphones running CommCare^® Application to collect demographic and clinical information. Data was stored in a secured cloud server. Clinical information collected included the range and duration of respiratory symptoms, presence of comorbidities, and reported history of exposure to confirmed COVID-19. Two nasopharyngeal swabs were collected by a surveillance officer (nurse, clinical officer, or laboratory technologist) trained to collect respiratory specimens from each of the nostrils a few seconds apart. The first specimen was collected using the swab provided in the Panbio^™ Ag RDT kit. The second specimen was collected using a polyester-tipped aluminum shafted swab and immediately placed in standard viral transport media prepared in the testing laboratory. The transport media was stored at +2 to +8°C before specimen collection and during shipment to the CDC-supported KEMRI laboratories in Nairobi or Kisumu for rRT-PCR testing. Aliquots of the specimens were then archived at -80°C.

Rapid SARS-CoV-2 antigen testing

Rapid testing was conducted using the Panbio^™ Ag RDT (Abbott Rapid Diagnostics, US. Ref. 41FK10) per manufacturer instructions [11] by surveillance officers. The Panbio^™ Ag RDT is a lateral flow immunoassay that detects the nucleocapsid (N) protein in nasal and nasopharyngeal specimens. Results were read and documented independently by two surveillance officers within 15 to 20 minutes. Positive, negative, and invalid results were interpreted according to the manufacturer’s recommendations. Each reader documented the test results in the electronic questionnaire independently. In the event the two readers documented different results, the final result was indicated as indeterminate.

Assessing the operational characteristics of the Panbio^™ Ag RDT kit

We assessed operational characteristics of the Panbio^™ Ag RDT in the field setting including clarity of kit instructions, technical complexity or ease of use, and the ease of interpretation of results via a standardized questionnaire administered to users on site.

rRT-PCR testing in the laboratory

According to the manufacturer’s instructions, total nucleic acid material was extracted from 200μL of the nasopharyngeal specimen using either the Ambion MagMax Total RNA isolation kit (Thermo Fisher Scientific, Greenville, North Carolina, USA) performed on a semi-automated KingFisher flex machine (Thermo Fisher Scientific) or Standard M Spin-X Viral RNA Extraction Kit (SD biosensor) according to the manufacturer’s instructions. We then eluted 60uL of total RNA from the extracted sample. The SARS-CoV-2 rRT-PCR was performed using the TaqPath 1-step Multiplex rRT-PCR master mix (Thermo Fisher Scientific) or using the GoTaq^® Probe 1-Step RT-qPCR System (Promega A6120). Both systems are compatible with the rRT-PCR CDC IDT kit. Each of the 15μL reaction volumes contained 5μL of master mix, 8.5μL of nuclease-free water, and 1.5μL of CDC-IDT primer/probe mix for each of the N1, N2, and RNP3 genes separately. The eluate was then amplified using QuantStudio^™ 5 Real-Time PCR System, 0.1ml block (Thermo Fisher Scientific), and analyses were done using QuantStudio 3 and 5 Real-Time PCR System Software version 1.5.1 (Thermo Fisher Scientific). SARS-CoV-2 assay targets, N1 and N2 (N1 and N2) were run simultaneously with the human ribonuclease P gene (RNP) control to monitor the quality of the nucleic acid extraction, specimen quality, and presence of reaction inhibitors for assay performance. The positive, negative, and human specimen controls were included in all assays.

When all controls exhibited expected performance, a positive result for SARS-CoV-2 was considered, if all assay amplification curves crossed the threshold line within Ct <40. If SARS-CoV-2 (N1 and N2) assay results were negative, then the test result was reported as negative. If only one target (N1 and N2) were positive, then it was designated as inconclusive, and retesting was conducted. If upon repeating the test, the results remain inconclusive, then the final result was reported as inconclusive. Results were relayed back to the specific sites for patient management.

Sample size calculation

To evaluate the diagnostic performance of the Panbio^™ Ag RDT device, we assumed a prevalence of 10% among symptomatic individuals and 5% in asymptomatic contacts of laboratory-confirmed persons based on current SARS CoV 2 local surveillance data. We assumed a sensitivity of 90% for the Panbio^™ Ag RDT with 7.5% margin of error, and after adjusting by 10% for other concerns we anticipated enrolling 770 symptomatic patients and 1,540 close contacts of confirmed cases.

Statistical analysis

Statistical analysis was conducted using Stata version 15.1 (College Station, Texas). Measures of central tendency and dispersion were calculated for continuous variables such as age, days since onset of symptoms, temperature, and cycle threshold (Ct) values. We calculated frequencies and proportions for categorical variables such as age group, occupation, Ct value cut-offs and the operational characteristics of the Ag RDT. The 95% confidence intervals for categorical variables were calculated using the normal approximation method (Wald’s test). We compared proportions between the symptomatic and asymptomatic groups by calculating their Z scores. P-values <0.05 were considered statistically significant.

We calculated the sensitivity, specificity, and positive and negative predictive values for the Panbio^™ Ag RDT against rRT-PCR as the reference standard. Stratified analysis of performance was done by age, day’s post-onset of symptoms, and Ct values. The measure of agreement between Ag RDT and rRT-PCR results was evaluated using Cohen’s Kappa statistic. We interpreted the strength of agreement as slight agreement (κ, 0–0.2), fair (κ, 0.21–0.4), moderate (κ, 0.41–0.6), substantial (κ, 0.61–0.8), and almost perfect (κ, 0.81–1.0) as described by Landis et al. [12].

Results

Enrolled participant characteristics

We enrolled 2,279 participants, where 38.9% had at least one respiratory symptom. The median age was 31.0 years (range <1 month– 103 years), and over half (60.9%) were male (Table 1). The median duration post-onset of symptoms among symptomatic participants was three days, with the majority (67.8%) presenting within 5 days.

[Figure omitted. See PDF.]

Laboratory findings

Panbio^™ Ag RDT test results.

Among the 2,277 tested using the Panbio^™ Ag RDT device, 12.4% were positive (Fig 1). There were no invalid or indeterminate Ag RDT results. The positivity was significantly higher among symptomatic individuals (18.7%) compared with the asymptomatic (8.4%,) individuals.

[Figure omitted. See PDF.]

Ag RDT, +ve, +ve, positive; -ve negative; antigen rapid diagnostic test; rRT-PCR–real time reverse transcriptase polymenrase chain reaction.

rRT PCR test results.

We tested 2,277 specimens by rRT-PCR, 24.2% tested positive and 1.4% were inconclusive (Fig 1). The overall median Ct value of the 551 rRT-PCR positive results was 30.2 (Interquartile range (IQR) 22.9–35.6). Symptomatic individuals had a significantly lower median Ct value of 26.2 (IQR 21.1–33.6) compared to 33.0 (IQR 25.0–36.3) among asymptomatic individuals (p <0.001). The median Ct value was significantly lower among those with Ag RDT positive results (23.2, IQR 19.8–27.1) compared to negative test results (35.1, IQR 32.0–36.9) (p <0.001) (Fig 2).

[Figure omitted. See PDF.]

Ag RDT, antigen rapid diagnostic test; Ct, cycle threshold; n, number of; rRT-PCR–real time reverse transcriptase polymenrase chain reaction.

Performance of Panbio^™ Ag RDT

We reviewed 2,245 paired Ag RDT/rRT-PCR records for the performance analysis. The overall sensitivity of the Panbio^™ Ag RDT was 46.6% (95% CI: 42.4–50.9%) and specificity was 98.5% (95% CI: 97.8–99.0%) (Table 2). At the observed SARS-CoV-2 prevalence (24.5%) the positive predictive value (PPV) was 90.8% (95% CI: 86.8–93.9%) and the negative predictive value (NPV) was 85.0% (95% CI: 83.4–86.6%) (Table 1; S1 and S2 Figs). The overall agreement between the Ag RDT and rRT-PCR results was moderate (k = 0.5,), substantial (k = 0.7) in those with symptoms, and fair (k = 0.4) among those without. The performance of the Panbio^™ Ag RDT is summarized in Table 2.

[Figure omitted. See PDF.]

In those with symptoms, false-negative results (Ag RDT-ve/rRT-PCR+ve) occurred in 100 (39.4%), and false positives(Ag RDT+ve/rRT-PCR-ve) occured in two percent (12/632) participants. Among the asymptomatic individuals, 65.3% (n = 194) had a false negative Ag RDT result, and 1.3% (n = 14) had a false positive result by Ag RDT.

Among the 876 symptomatic participants, the sensitivity was 60.6% (95% CI: 54.3–66.7%) (Table 2). In those with onset of symptoms <5 days sensitivity was 67.1%, declining to 53.3% among those with onset >7 days. Sensitivity in asymptomatic participants,was significantly lower at 34.7% (95% CI: 29.3–40.4%) compared to symptomatic ones (p <0.001). Specificity remained high in symptomatic (98.1%, 95% CI: 96.7–99.0%) and asymptomatic individuals (98.7%, 95% CI: 96.7–99.0%) (S4 Fig).

Regardless of the presence or absence of symptoms, the overall sensitivity of the Ag RDT test in 273 specimens with Ct values ≤ 30 was 81.3% (95% CI: 72.2–91.6). This reduced to 12.6% (95% CI: 11.2–14.2%) among the 278 individuals with Ct >30 (Table 2).

Evaluation of the user’s experience of the Panbio^™ Ag RDT

A total of 17 surveillance officers comprising 10 (59%) clinicians and 7 (41%) laboratory staff completed the survey. All 17(100%) surveillance officers found the test procedure easy to perform, and the manufacturers’ instructions were found to be clear and easy to follow. Four (23.5%) of the surveillance officers noted the difficulty in dispensing a sample onto the test device from the extraction tube when nasopharyngeal specimens were collected from individuals with thick mucus.

Discussion

We present findings of a field evaluation of the Panbio^™ Ag RDT conducted among symptomatic patients with onset of symptoms up to 14 days and asymptomatic individuals who had contact with individuals with confirmed SARS-CoV-2 infection within two days before detection of the index case and up to 14 days after exposure. Our evaluation was conducted in the third COVID-19 wave in Kenya, however, the majority of our participants were enrolled towards the end of the wave [13,14] where the median national prevalence was 9.6%(IQR 7.9–13.1%) [15]. The proportion of symptomatic individuals enrolled (54%) was higher than observed in the Kenyan population [13].

The overall sensitivity was low (46.6%), detecting slightly less than half of the rRT-PCR confirmed cases, therefore, missing more than half and classifying them as not having SARS-CoV-2 infection when they were indeed infected. In our evaluation, sensitivity varied with the presence or absence of symptoms, days since onset of onset, and viral load. As expected, sensitivity was higher among symptomatic individuals compared to asymptomatic ones (60.6% vs. 34.7%), and those with higher viral loads compared to those with lower ones (Ct value ≤30 vs. >30). Sensitivity increased with more recent onset of infection, being higher in those with onset of symptoms less than 7 days compared to those with onset >7 days (61.6% vs. 47.1%). By duration since onset of infection, sensitivity was highest among those with onset between 3 and 5 days. Symptomatic individuals had significantly higher viral loads (Fig 2), and with the highest sensitivity (87.0%) observed where Ct values ≤30, exceeding the WHO recommendation.

The low overall sensitivity observed may be due to the high proportion (61.1%) of asymptomatic participants, who had significantly higher Ct values and therefore more likely to test negative by Ag RDT. Antigen RDTs perform better in individuals with higher viral loads [7]. Sensitivity was lower than indicated in the manufacturer’s product insert (61.6% vs. 91.1%) [11] and the WHO minimum recommendation (80%) [8]. This is likely due to the selection of specimens with high viral loads by the manufacturer’s study for approval as hypothesized by Albert et al., 2021 in an evaluation where SARS-CoV-2 cultures were conducted [4].

The overall sensitivity was lower than reported in similar studies conducted in a multicenter study in Spain conducted among individuals of any age (57.3%, 95%CI: 48.3–95.8%) [16], Spanish adults in primary care centers (71.4%, 95% CI: 63.1–78.7%) [17], among Spanish individuals of any age (73.3%, 95% CI: 62.2–83.8%) [18], and those presenting within 7 days of onset or history of exposure (90.5%, 95% CI 87.5–93.6%) [19]. The lower sensitivity we observed could have been due to the relatively younger particpants (median age 31 years) enrolled in our evaluation compared to other studies with higher median age (39–51.5 years) [18,20]. A recent study demonstrated increasing viral load with increasing age [21] and resultant higher sensitivities. In addition, while we did not ascertain the variants in the specimens tested, the alpha variant and delta variants predominated the third wave in Kenya [22,23]. While some literature indicates that variants of concern (17,18) do not reduce the sensitivity and specificity of Ag RDTs, a recent study reported that sensitivity of the Panbio^™ Ag RDT declined significantly in individuals infected with the alpha variant (53.0%) compared to those with non-alpha variants (89.0%) even after adjusting for viral load (p <0.002) [24] as was also demonstrated by Wertenauer et. al in the COVAG study [25]. Mutations occurring in the N protein of the Alpha variant may not be detected by the Panbio^™ Ag RDT [26]. Laboratory studies have demonstrated a tenfold reduction in detection of the alpha variant by the Panbio^™ Ag RDT [27]. Early laboratory data also demonstrated reduced sensitivity of other Ag RDTs in laboratory studies [28–30], while others have demonstrated no impact on performance [31]. Further clinical evaluations may be required to determine the impact of viral mutations on the performance on Ag RDTs.

The sensitivity observed among symptomatic individuals was similar to that observed in symptomatic Mexican adults (58.1%, 95% CI: 54.9–61.3%) [32]. However, it was higher than reported in a multicenter evaluation conducted among symptomatic children (60.6% vs. 45.4%) with the onset of symptoms within five days [20].

Sensitivity was higher among symptomatic individuals compared to asymptomatic ones. Similar findings were observed in other evaluations with higher sensitivity among symptomatic individuals compared to asymptomatic in a study in Mexico (58.1% vs. 26.3%) [32], in Spain (80.4% vs. 56.6%) [17], among Greek children (95.2% vs. 22.2%) [33], and among Swiss children (73% vs. 43%) [34]. However, a German study found similar sensitivities between the two groups (86.8% vs. 85.7%) [35]. The symptomatic individuals we enrolled in our evaluation had significantly lower Ct values than asymptomatic participants (Fig 2). Our evaluation further observed that irrespective of symptoms sensitivity was higher among those with Ct value ≤30 as reported by other authors [17,35–37]. We also observed significantly higher viral loads among symptomatic individuals compared to asymptomatic individuals (p <0.001) as has reported in recent literature [38,39]. In contrast, some studies found no difference in viral load between the two groups [40,41] while others found higher viral load among asymptomatic individuals [42,43]. Higher sensitivity has been observed among individuals presenting in the early symptomatic phase of their infection [19,32,35–37,42,44,45].

While rRT-PCR testing for SARS-CoV-2 is the reference standard for diagnosis, its limited access in our setting and provision of results in a clinically relevant timely manner to inform case management limits its clinical utility. Therefore, given reasonable sensitivity, individuals with a negative Ag RDT test in an individual meeting the clinical and epidemiological criteria may require a confirmatory test by rRT-PCR. The variation in sensitivity by presence or absence of symptoms, symptom duration, and viral load indicates that the kit may only be useful among individuals with high viral loads, especially those with symptoms.

The PPV (90.8%) observed at the prevalence of SARS-CoV-2 infection of 24.5% by rRT-PCR was similar to the projected PPV (89.6%.) assuming the minimum WHO recommended sensitivity (80%) and specificity (97.0%). The NPV (85.0%) was lower compared to that computed NPV (93.7%) assuming the minimum recommendations. Among individuals who tested negative by Ag RDT, the likelihood of being wrongly classified as uninfected was 15%. In 2021, the median positivity rate in Kenya was 7.4% (IQR, 3.2–13.1%) [46]. At the observed sensitivity and specificity and the national positivity rate, PPV would range between 50.2% and 82.8%, and the NPV 92.4%–98.2%).

The overall specificity remained high (98.5%) and was above the WHO recommendation [47] for both symptomatic and asymptomatic individuals and across the other subgroups (Table 2) as has been reported in other evaluations [32,35,36,44,48–51]. The Ag RDT had high specificity with less than (1.5%) of the tests (n = 26) giving a false-positive result. This small proportion represents individuals who would be falsely classified as infected and unnecessarily be managed as COVID-19 cases or self-isolated and their contacts traced and quarantined.

However, we observed over half (53%) of the Ag RDT results gave false-negative results with a higher proportion (65%) among the asymptomatic individuals. This observation emphasizes the recommendation by WHO to re-test symptomatic patients with rRT-PCR when they receive a negative Ag RDT result, especially in settings where SARS-CoV-2 prevalence is ≤ 5%. Further, as a screening tool for testing contacts of confirmed cases, the 65% false negatives among asymptomatic contacts represent the proportion that would falsely be classified as virus-free, not self-isolate, and potentially infect others.

The Panbio^™ Ag RDT was found to be easy to use by the majority of the users with a quick TAT for clinical decision-making and implementation of preventive measures to contain transmission as observed in the literature [35,36]. The main challenge we observed in conducting Ag RDT testing using the Panbio^™ Ag RDT kits was dispensing the specimen onto the test device in samples collected from individuals with thick mucus. This was, however, solved by ensuring that individuals blew their noses before specimen collection.

Our field evaluation had several strengths. The evaluation was conducted in the real-world setting across multiple sites under the point-of-care conditions which had a large sample size comprising two groups (symptomatic and asymptomatic individuals) thereby representing individuals at any point of the COVID-19 disease spectrum.

Limitation

The number of persons enrolled in the two groups, symptomatic and asymptomatic, may not reflect the true prevalence in the general population as we evaluated the Ag RDT during a period of high COVID-19 prevalence. While rRT-PCR was used as the reference standard, it is not a gold standard as it may not truly differentiate replicating virus and residual RNA from recent infection among those with high Ct values. Residual RNA may not be viable and therefore not infectious [52].

Conclusion

The overall sensitivity and PPV of the Panbio^™ Ag RDT were much lower than expected. Sensitivity was acceptable in symptomatic individuals with Ct value ≤30. The specificity of the Ag RDT was high and satisfactory; therefore, a positive result may not require confirmation by rRT-PCR. The kit may be useful as a rapid screening tool for only symptomatic patients in high-risk settings with limited access to rRT-PCR. A negative result should be interpreted based on clinical and epidemiological information and may require retesting by rRT-PCR.

Supporting information

S1 Fig. The positive predictive value of the Panbio^™ Ag RDT by clinical status and days post onset of symptoms, Kenya 2021.

https://doi.org/10.1371/journal.pone.0277657.s001

(TIF)

S2 Fig. The negative predictive value of the Panbio^™ Ag RDT by clinical status and days post onset of symptoms, Kenya 2021.

https://doi.org/10.1371/journal.pone.0277657.s002

(TIF)

S3 Fig. The sensitivity of the Panbio^™ Ag RDT by clinical status, and Ct values.

https://doi.org/10.1371/journal.pone.0277657.s003

(TIF)

S4 Fig. The specificity of the Panbio^™ Ag RDT by clinical status and days post onset of symptoms, Kenya 2021.

https://doi.org/10.1371/journal.pone.0277657.s004

(TIF)

Acknowledgments

We acknowledge all the study participants, surveillance officers, and laboratory staff that participated in this evaluation. In addition, we thank the Ministry of Health Kenya, the Kenya Prisons Service, Washington State University (WSU) Global Health Kenya, and the County departments of Health of Kiambu, Mombasa, Nakuru, Nyeri, and Siaya for the continued support throughout the implementation of the evaluation.

Citation: Irungu JK, Munyua P, Ochieng C, Juma B, Amoth P, Kuria F, et al. (2023) Diagnostic accuracy of the Panbio COVID-19 antigen rapid test device for SARS-CoV-2 detection in Kenya, 2021: A field evaluation. PLoS ONE 18(1): e0277657. https://doi.org/10.1371/journal.pone.0277657

About the Authors:

Jack Karuga Irungu

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Field Epidemiology and Laboratory Training Program, Ministry of Health, Nairobi, Kenya

ORICD: https://orcid.org/0000-0001-9000-5144

Peninah Munyua

Roles: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

ORICD: https://orcid.org/0000-0003-1950-8616

Caroline Ochieng

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation: Kenya Medical Research Institute (KEMRI), Center for Global Health Research, Nairobi, Kenya

Bonventure Juma

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Patrick Amoth

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: Ministry of Health, Nairobi, Kenya

Francis Kuria

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: Ministry of Health, Nairobi, Kenya

John Kiiru

Roles: Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: Ministry of Health, Nairobi, Kenya

Lyndah Makayotto

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: Ministry of Health, Nairobi, Kenya

Ahmed Abade

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation: Field Epidemiology and Laboratory Training Program, Ministry of Health, Nairobi, Kenya

Marc Bulterys

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Elizabeth Hunsperger

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Gideon O. Emukule

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Clayton Onyango

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Taraz Samandari

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

ORICD: https://orcid.org/0000-0003-4830-6108

Beth A. Tippett Barr

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Victor Akelo

Roles: Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Herman Weyenga

Roles: Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

ORICD: https://orcid.org/0000-0002-1111-1764

Patrick K. Munywoki

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Godfrey Bigogo

Roles: Conceptualization, Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: Kenya Medical Research Institute (KEMRI), Center for Global Health Research, Nairobi, Kenya

Nancy A. Otieno

Roles: Conceptualization, Investigation, Methodology, Writing – original draft, Writing – review & editing

Affiliation: Kenya Medical Research Institute (KEMRI), Center for Global Health Research, Nairobi, Kenya

ORICD: https://orcid.org/0000-0002-7083-4735

Jackton Azenga Kisivuli

Roles: Conceptualization, Investigation, Methodology, Project administration, Resources, Validation, Writing – original draft, Writing – review & editing

Affiliation: Directorate of Health, Kenya Prisons Service, Nairobi, Kenya

Edwin Ochieng

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Writing – original draft, Writing – review & editing

Affiliation: Association of Public Health Laboratories (APHL), Nairobi, Kenya

Rufus Nyaga

Roles: Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: Association of Public Health Laboratories (APHL), Nairobi, Kenya

Noah Hull

Roles: Conceptualization, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation: Association of Public Health Laboratories (APHL), Nairobi, Kenya

ORICD: https://orcid.org/0000-0002-1710-8929

Amy Herman-Roloff

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: U.S. Centers for Disease Control and Prevention, Nairobi, Kenya

Rashid Aman

Roles: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliation: Ministry of Health, Nairobi, Kenya

References

1. United Nations, “A UN framework for the immediate response to Table of Contents,” United Nations, no. April. 2020.

2. World Health Organization (WHO), “Recommendations for national SARS-CoV-2 testing strategies and diagnostic capacities,” no. June. pp. 1–16, 2021.

3. World Health Organization (WHO), “COVID-19 Target product profiles for priority diagnostics to support response to the COVID-19 pandemic v.1.0,” no. September, pp. 1–38, 2020.

4. Albert E. et al., “Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres,” Clin. Microbiol. Infect., vol. 27, no. 3, pp. 472.e7–472.e10, Mar. 2021.

5. Barra G. B., Rita T. H. S., Mesquita P. G., Jácomo R. H., and Nery L. F. A., “Overcoming supply shortage for SARS-CoV-2 detection by RT-qPCR,” Genes (Basel)., vol. 12, no. 1, pp. 1–10, 2021. pmid:33450867

6. Duma Z. et al., “The challenges of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) testing in low-middle income countries and possible cost-effective measures in resource-limited settings,” Global. Health, vol. 18, no. 1, pp. 1–10, 2022.

7. W. World Health Organization, “Antigen-detection in the diagnosis of SARS-CoV-2 infection—October 2021,” no. October. 2021.

8. World Health Organization (WHO), “Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays Interim guidance, 11 September 2020.,” World Heal. Organ., no. September, pp. 1–9, 2020.

9. World Health Organization, “Antigen-detection in the diagnosis of SARS-CoV-2 infection Interim guidance,” 2021.

10. World Health Organization (WHO), “WHO Emergency Use Listing for In vitro diagnostics (IVDs) Detecting SARS-CoV-2. Geneva,” World Health Organization (WHO)., no. September. Geneva, 2020.

11. Abbott, “Panbio™ Test Device COVID-19 Ag Rapid -In vitro diagnostic rapid test for qualitative detection of SARS-CoV-2 antigen (Ag).” 2021.

12. Landis J. R. and Koch G. G., “The Measurement of Observer Agreement for Categorical Data,” Biometrics, vol. 33, no. 1, p. 159, Mar. 1977. pmid:843571

13. Ministry of Health Kenya, “Covid-19 Outbreak in Kenya Situation report 670,” 2021.

14. W. World Health Organization, “WHO Coronavirus Disease (COVID-19) Dashboard With Vaccination Data | WHO Coronavirus (COVID-19) Dashboard With Vaccination Data,” World Health Organization, 2022. [Online]. https://covid19.who.int/region/afro/country/ke. [Accessed: 20-Feb-2022].

15. L. R.-G. C. A. C. G. E. O.-O. J. H. B. M. D. B. Hannah Ritchie Edouard Mathieu and M. Roser, “Coronavirus Pandemic (COVID-19),” Our World in Data, 2020. [Online]. https://ourworldindata.org/coronavirus.

16. Agulló V. et al., “Evaluation of the rapid antigen test Panbio COVID-19 in saliva and nasal swabs in a population-based point-of-care study,” J. Infect., vol. 82, no. 5, pp. 186–230, 2021.

17. Bulilete O. et al., “Panbio™ rapid antigen test for SARS-CoV-2 has acceptable accuracy in symptomatic patients in primary health care,” J. Infect., vol. 82, no. 3, pp. 391–398, Mar. 2021.

18. Linares M. et al., “Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms,” J. Clin. Virol., vol. 133, no. January, p. 104659, Dec. 2020. pmid:33160179

19. Merino P. et al., “Multicenter evaluation of the Panbio™ COVID-19 rapid antigen-detection test for the diagnosis of SARS-CoV-2 infection,” Clin. Microbiol. Infect., vol. 27, no. 5, pp. 758–761, 2021.

20. Villaverde S. et al., “Diagnostic Accuracy of the Panbio Severe Acute Respiratory Syndrome Coronavirus 2 Antigen Rapid Test Compared with Reverse-Transcriptase Polymerase Chain Reaction Testing of Nasopharyngeal Samples in the Pediatric Population,” J. Pediatr., vol. 232, no. January, pp. 287–289.e4, May 2021. pmid:33484697

21. Euser S. et al., “SARS-CoV-2 viral-load distribution reveals that viral loads increase with age: a retrospective cross-sectional cohort study,” Int. J. Epidemiol., vol. 50, no. 6, pp. 1795–1803, Jan. 2022. pmid:34999848

22. KEMRI, “Genomic surveillance detects local transmission of the global variants of concern in nine counties in Kenya,” 2021.

23. G. Kimita et al., “A genomics dissection of Kenya’s COVID-19 waves: temporal lineage replacements and dominance of imported variants of concern,” 2021.

24. van Ogtrop M. L., van de Laar T. J. W., Eggink D., Vanhommerig J. W., and van der Reijden W. A., “Comparison of the Performance of the PanBio COVID-19 Antigen Test in SARS-CoV-2 B.1.1.7 (Alpha) Variants versus non-B.1.1.7 Variants,” Microbiol. Spectr., vol. 9, no. 3, pp. 1–22, Dec. 2021. pmid:34817226

25. Wertenauer C. et al., “Diagnostic Performance of Rapid Antigen Testing for SARS-CoV-2: The COVid-19 AntiGen (COVAG) study,” Front. Med., vol. 9, no. March, Mar. 2022. pmid:35386920

26. Jian M. J. et al., “SARS-CoV-2 variants with T135I nucleocapsid mutations may affect antigen test performance,” Int. J. Infect. Dis., vol. 114, pp. 112–114, 2022. pmid:34758391

27. Raïch-Regué D. et al., “Performance of SARS-CoV-2 Antigen-Detecting Rapid Diagnostic Tests for Omicron and Other Variants of Concern,” Front. Microbiol., vol. 13, no. May, pp. 1–6, May 2022. pmid:35620108

28. Stanley S. et al., “Limit of Detection for Rapid Antigen Testing of the SARS-CoV-2 Omicron and Delta Variants of Concern Using Live-Virus Culture,” J. Clin. Microbiol., vol. 60, no. 5, May 2022. pmid:35440165

29. U.S. Food and Drug Administration (FDA), “SARS-CoV-2 Viral Mutations: Impact on COVID-19 Tests,” FDA. 2021.

30. Barrera-Avalos C. et al., “The Rapid Antigen Detection Test for SARS-CoV-2 Underestimates the Identification of COVID-19 Positive Cases and Compromises the Diagnosis of the SARS-CoV-2 (K417N/T, E484K, and N501Y) Variants,” Front. Public Heal., vol. 9, no. January, pp. 1–8, Jan. 2022. pmid:35047474

31. Bekliz M. et al., “SARS-CoV-2 antigen-detecting rapid tests for the delta variant,” The Lancet Microbe, vol. 3, no. 2, p. e90, Feb. 2022. pmid:34849494

32. Thirion-Romero I. et al., “Evaluation of Panbio rapid antigen test for SARS-CoV-2 in symptomatic patients and their contacts: a multicenter study,” Int. J. Infect. Dis., vol. 113, pp. 218–224, Dec. 2021. pmid:34678504

33. Eleftheriou I. et al., “Real-life evaluation of a COVID-19 rapid antigen detection test in hospitalized children,” J. Med. Virol., vol. 93, no. 10, pp. 6040–6044, 2021. pmid:34156112

34. L’Huillier A. G. et al., “Diagnostic accuracy of sars-cov-2 rapid antigen detection testing in symptomatic and asymptomatic children in the clinical setting,” J. Clin. Microbiol., vol. 59, no. 9, pp. 1–8, 2021. pmid:34190574

35. Krüger L. J. et al., “The Abbott PanBio WHO emergency use listed, rapid, antigen-detecting point-of-care diagnostic test for SARS-CoV-2—Evaluation of the accuracy and ease-of-use,” PLoS One, vol. 16, no. 5, p. e0247918, May 2021. pmid:34043631

36. FIND, “FIND Evaluation of Abbott Panbio COVID-19 Ag Rapid Test Device External Report,” no. December, pp. 1–4, 2020.

37. Alemany A. et al., “Analytical and clinical performance of the panbio COVID-19 antigen-detecting rapid diagnostic test,” J. Infect., vol. 82, no. 5, pp. 186–230, May 2021. pmid:33421447

38. Almadhi M. A. et al., “The high prevalence of asymptomatic SARS-CoV-2 infection reveals the silent spread of COVID-19,” Int. J. Infect. Dis., vol. 105, pp. 656–661, 2021. pmid:33647516

39. Strutner J. et al., “Comparison of Reverse-Transcription Polymerase Chain Reaction Cycle Threshold Values From Respiratory Specimens in Symptomatic and Asymptomatic Children With Severe Acute Respiratory Syndrome Coronavirus 2 Infection,” Clin. Infect. Dis., vol. 73, no. 10, pp. 1790–1794, Nov. 2021. pmid:34060629

40. Zuin M., Gentili V., Cervellati C., Rizzo R., and Zuliani G., “Viral load difference between symptomatic and asymptomatic COVID-19 patients: Systematic review and meta-analysis,” Infect. Dis. Rep., vol. 13, no. 3, pp. 645–653, 2021. pmid:34287354

41. Lavezzo E. et al., “Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo’,” Nature, vol. 584, no. 7821, pp. 425–429, Aug. 2020.

42. Cardillo L. et al., “SARS-CoV-2 quantitative real time PCR and viral loads analysis among asymptomatic and symptomatic patients: an observational study on an outbreak in two nursing facilities in Campania Region (Southern Italy),” Infect. Agent. Cancer, vol. 16, no. 1, pp. 1–7, 2021.

43. Hasanoglu I. et al., “Higher viral loads in asymptomatic COVID-19 patients might be the invisible part of the iceberg,” Infection, vol. 49, no. 1, pp. 117–126, 2021. pmid:33231841

44. Berger A. et al., “Diagnostic accuracy of two commercial SARSCoV- 2 antigen-detecting rapid tests at the point of care in community-based testing centers,” PLoS One, vol. 16, no. 3 March 2021, pp. 1–12, 2021.

45. Ferté T. et al., “Accuracy of COVID-19 rapid antigenic tests compared to RT-PCR in a student population: The StudyCov study,” J. Clin. Virol., vol. 141, no. January, p. 104878, Aug. 2021.

46. Hasell J. et al., “A cross-country database of COVID-19 testing,” Sci. Data, vol. 7, no. 1, p. 345, Dec. 2020. pmid:33033256

47. World Health Organization, “SARS-CoV-2 Antigen detecting rapid diagnostic test implementation projects,” 2020. [Online]. https://www.who.int/news-room/articles-detail/sars-cov-2-antigen-detecting-rapid-diagnostic-test-implementation-projects.

48. Albert E. et al., “Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres,” Clin. Microbiol. Infect., vol. 27, no. 3, pp. 472.e7–472.e10, 2021.

49. A. Christensen et al., “Evaluation of Abbot’s Panbio COVID -19 rapid antigen test in Norway,” 2020.

50. Torres I., Poujois S., Albert E., Colomina J., and Navarro D., “Evaluation of a rapid antigen test (Panbio™ COVID-19 Ag rapid test device) for SARS-CoV-2 detection in asymptomatic close contacts of COVID-19 patients,” Clin. Microbiol. Infect., vol. 27, no. 4, pp. 636.e1–636.e4, Apr. 2021.

51. Dinnes J. et al., “Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection,” Cochrane Database Syst. Rev., vol. 2021, no. 4, Mar. 2021.

52. Carmo A. et al., “Clearance and persistence of SARS-CoV-2 RNA in patients with COVID-19,” J. Med. Virol., vol. 92, no. 10, pp. 2227–2231, Oct. 2020. pmid:32484958