Performance of diagnostic and predictive host blood transcriptomic signatures for Tuberculosis disease: A systematic review and meta-analysis

Abstract

Introduction

Host blood transcriptomic biomarkers have potential as rapid point-of-care triage, diagnostic, and predictive tests for Tuberculosis disease. We aimed to summarise the performance of host blood transcriptomic signatures for diagnosis of and prediction of progression to Tuberculosis disease; and compare their performance to the recommended World Health Organisation target product profile.

Methods

A systematic review and meta-analysis of the performance of host blood mRNA signatures for diagnosing and predicting progression to Tuberculosis disease in HIV-negative adults and adolescents, in studies with an independent validation cohort. Medline, Scopus, Web of Science, and EBSCO libraries were searched for articles published between January 2005 and May 2019, complemented by a search of bibliographies. Study selection, data extraction and quality assessment were done independently by two reviewers. Meta-analysis was performed for signatures that were validated in ≥3 comparable cohorts, using a bivariate random effects model.

Results

Twenty studies evaluating 25 signatures for diagnosis of or prediction of progression to TB disease in a total of 68 cohorts were included. Eighteen studies evaluated 24 signatures for TB diagnosis and 17 signatures met at least one TPP minimum performance criterion. Three diagnostic signatures were validated in clinically relevant cohorts to differentiate TB from other diseases, with pooled sensitivity 84%, 87% and 90% and pooled specificity 79%, 88% and 74%, respectively. Four studies evaluated signatures for progression to TB disease and performance of one signature, assessed within six months of TB diagnosis, met the minimal TPP for a predictive test for progression to TB disease.

Conclusion

Host blood mRNA signatures hold promise as triage tests for TB. Further optimisation is needed if mRNA signatures are to be used as standalone diagnostic or predictive tests for therapeutic decision-making.

About the Authors:

Humphrey Mulenga

Roles Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

ORCID logo http://orcid.org/0000-0002-2030-0751

Chambrez-Zita Zauchenberger

Roles Formal analysis, Investigation, Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

Erick W. Bunyasi

Roles Investigation, Methodology, Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

Stanley Kimbung Mbandi

Roles Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

Simon C. Mendelsohn

Roles Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

ORCID logo http://orcid.org/0000-0002-4054-2766

Benjamin Kagina

Roles Validation, Writing – review & editing

Affiliation: Vaccines for Africa Initiative (VACFA), School of Public Health & Family Medicine, University of Cape Town, Cape Town, South Africa

Adam Penn-Nicholson

Roles Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

Thomas J. Scriba

Contributed equally to this work with: Thomas J. Scriba, Mark Hatherill

Roles Conceptualization, Methodology, Supervision, Writing – review & editing

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

Mark Hatherill

Contributed equally to this work with: Thomas J. Scriba, Mark Hatherill

Roles Conceptualization, Methodology, Supervision, Writing – review & editing

* E-mail: [email protected]

Affiliation: South African Tuberculosis Vaccine Initiative (SATVI), Institute of Infectious Disease & Molecular Medicine and Division of Immunology, Department of Pathology, University of Cape Town, Cape Town, South Africa

Introduction

The World Health Organisation (WHO) has targeted 2035 to end tuberculosis (TB) and aims for 90% reduction in new TB cases and 95% reduction in TB deaths compared to 2015 levels [1]. Non-sputum triage, diagnostic, and predictive tests for TB may play a role in advancing TB control efforts. The WHO, in conjunction with the Foundation for Innovative New Diagnostics and the New Diagnostics Working Group of the Stop TB Partnership, has published Target Product Profiles (TPPs) for non-sputum biomarker triage, diagnostic, and predictive tests for progression from latent TB infection (LTBI) to TB disease [2–4]. The TPPs require minimum 90% sensitivity and 70% specificity for a triage test; 65% sensitivity and 98% specificity for a diagnostic test [2], and 75% sensitivity and 75% specificity for a test to predict progression from LTBI to active TB disease within two years [3, 4]. A new predictive test should also achieve a positive predictive value (PPV) of 5.8% given a 2% pre-test probability [3].

Current commercially available TB diagnostic tests are not optimal. Mycobacterium tuberculosis (MTB) culture, considered the gold standard, requires days to weeks to obtain a result from a reference laboratory, and is thus not ideal for rapid patient management [5, 6].Sputum smear microscopy has low sensitivity [7] ranging from 32% to 89% [8], resulting in a considerable proportion of active pulmonary TB patients being missed [9]. Sputum Xpert MTB/RIF and Xpert MTB/RIF Ultra have considerably better diagnostic performance [10, 11] than smear microscopy, but are similarly dependant on obtaining an adequate sputum sample; and need specialised laboratory equipment and a reliable power supply, which impedes routine screening in TB-endemic resource-limited settings [12].

Individuals with LTBI, defined by a positive tuberculin skin test (TST) or interferon-gamma release assay (IGRA), have a higher risk of progression to TB disease than MTB-uninfected people [13, 14]. However, only about 10–15% of people who test IGRA or TST positive will go on to develop TB disease [15, 16]. Predictive specificity of IGRA and TST for incident TB disease is poor (49.3% and 45% respectively) [14, 17]. While prevention of TB disease arising from LTBI is key to achieving WHO elimination targets [18], mass preventive therapy based on IGRA or TST screening in TB-endemic countries would need to treat a significant proportion of the population [19], most of them unnecessarily, which would be unaffordable and potentially ineffective, because re-infection would likely occur before programmatic coverage was complete.

In recent years, host blood transcriptomic signatures have offered a promising alternative as tests for both diagnosis of and prediction of progression to TB disease. These signatures have also improved our understanding of inflammatory processes associated with progression [20, 21] in individuals with MTB infection [22] and those with TB disease [23]. Host blood transcriptomic signatures have been shown to discriminate prevalent TB disease cases from MTB-uninfected and latently MTB-infected individuals, and individuals with other respiratory ailments [24]; and predict progression to TB disease in individuals with LTBI [25, 26].

Two systematic reviews have evaluated biomarkers for diagnosis of TB disease in children [27] and all age groups [28]. These systematic reviews did not include biomarkers for progression to TB disease; and did not include a meta-analysis, owing to heterogenous study designs, patient selection, and biomarker composition. Two additional studies performed re-analysis of patient-level data but did not perform a systematic review of signature performance as reported by the original studies. Warsinske et al [29] compared 16 signatures for TB diagnosis by recreating the original model of each signature and evaluated each signature across the datasets they had identified, and Gupta et al [30] conducted a meta-analysis of patient-level pooled data for signatures of incipient TB. We present a systematic review and meta-analysis of transcriptomic signatures which have been evaluated in independent validation cohorts. We aimed to summarise the performance of host blood transcriptomic signatures for diagnosis of and prediction of progression to TB disease in HIV-negative adults and adolescents; and to compare individual signature performance to the WHO TPP. This review is registered with the International Prospective Register of Systematic Reviews (PROSPERO), registration number CRD42017073817.

Materials and methods

We conducted a systematic review according to standard guidelines [31, 32] (S1 Table). and designed the protocol prior to conducting the review [33] (http://dx.doi.org/10.1136/bmjopen-2018-026612).

Study inclusion/exclusion criteria

We included studies evaluating host blood mRNA signatures for diagnosis of or prediction of progression to TB disease in HIV-negative adults and/or adolescents (≥ 12 years old) and published in English. Studies were restricted to those published between January 2005 and May 2019 to concentrate on recent evidence because of the fast pace at which the field of transcriptomics is changing and advancing. Only studies comparing TB disease cases versus MTB-uninfected controls, individuals with other diseases (ODs), or with LTBI; and using a microbiological reference standard of either sputum MTB culture, Xpert MTB/RIF, or smear microscopy for TB disease diagnosis, were eligible for inclusion. Studies of prediction of incident TB disease were required to be prospective, with a follow-up period of at least six months; and enrolling either TB contacts, latently MTB-infected, or uninfected individuals. Studies were excluded if: conducted in animals; were in children younger than 12 years, did not report sensitivity and specificity, did not allow recreation of a 2 x 2 contingency table for calculation of test performance, and where authors did not respond to enquiries for data within four weeks of inquiry. Unpublished reports and conference proceedings were excluded due to absence of peer review and difficulty in obtaining data.

Literature search

Medline via PubMed, Scopus, Web of Science, and EBSCO databases were searched for relevant studies. The search strategy developed on PubMed was adapted to other databases and was as follows:

((((((Tuberculosis [MeSH] OR Mycobacterium tuberculosis [MeSH] OR (Tuberculosis OR TB OR Mycobacterium tuberculosis OR MTB))) AND ((Diagnosis [MeSH] OR Diagnosis [subheading] OR Prognosis [MeSH] OR (Diagnosis OR diagnostic OR detect* OR predict* OR prognosis OR prognostic OR screen*)))) AND ((Biomarkers/Blood [MeSH] OR RNA/Blood [MeSH] OR Transcription, Genetic [MeSH] /etiology/genetics/immunology OR (Blood Biomarker OR blood biomarkers OR bio-signature OR gene expression OR genetic transcription OR host blood OR immune marker OR immunologic marker OR Ribonucleic Acid OR RNA OR signature OR surrogate endpoint OR surrogate marker OR transcriptome OR transcriptomic)))) AND ((Area under Curve [MeSH] OR Sensitivity and Specificity [MeSH] OR (Area under curve OR area under curves OR AUC OR receiver operating characteristic OR ROC OR Accuracy OR Performance OR sensitivity OR specificity)))) AND (Humans[Mesh])) AND (("2005/01/01"[Date—Publication]: "2019/05/31"[Date—Publication])). Additionally, bibliographies of included papers were scrutinised for potential papers that were missed by the search terms.

Study selection

Two reviewers (HM and CZZ) independently screened search outputs for eligible studies. Publications were first screened by title and abstract, and thereafter by full text. Articles were independently categorised as either (i) selected, (ii) not selected, or (iii) pending. The two reviewers conferred to resolve any disagreements about pending publications, and if a consensus could not be reached, discrepancies were adjudicated by a third reviewer (BK). Fig 1 depicts the study selection process.

[Figure omitted. See PDF.]

Fig 1. Flow of studies in the review of transcriptomic signatures for diagnosing and predicting progression to TB disease.

DTA; diagnostic test accuracy, LTBI; Latent TB infection, OD; Other diseases.

https://doi.org/10.1371/journal.pone.0237574.g001

Data extraction and management

Data metrics were extracted separately from the relevant articles and double-entered into a Microsoft SQL Server 2012 database to facilitate electronic comparison of data between reviewers. For each biomarker test, we extracted the reported performance data; sensitivity, specificity, true positives (TP) false positives (FP), true negatives (TN), and false negatives (FN); as well as details of the study design, population, and index and reference test characteristics using a customised data extraction form (S1 File). We requested missing data from authors by email.

Appraisal of methodological quality of studies

HM and CZZ independently assessed the quality of included studies using a customised form based on the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) [34] assessment tool (S2 File, Methods in S3 File). BMK provided adjudication where a disagreement occurred. For each of the four domains in QUADAS-2, namely “patient selection”, “index test”, “reference standard” and “flow and timing”; risk of bias was scored as “low” if all responses in that domain were answered as “yes”, “high” if any of the responses were answered as “no” or “unclear” and “unclear” if it was unclear for all the responses. We judged “low applicability concerns” for the patient selection domain if a clinically relevant control population was used; OD for diagnostic studies and LTBI or household TB contacts for predictive studies and “high applicability concerns” if other populations were used. For other domains, we judged “low applicability concerns” if all signalling questions in that domain were answered as “yes”, “high applicability concerns” if any question was answered as “no” and “unclear applicability concerns” if answered as such.

GRADE quality of evidence

We used the “Grading of Recommendations Assessment, Development and Evaluation” (GRADE) approach to judge the quality of evidence. Classification of the quality of evidence was based on study design in conjunction with the five factors that affect study quality; study limitations, indirectness, inconsistency, imprecision, and publication bias [35].

Data synthesis and analysis

We extracted data to construct 2 x 2 contingency tables of reference test versus index test results. TB positive and TB negative were defined as participants with and without TB disease respectively, based on the reference standard. Forest plots of sensitivity and specificity with 95% confidence intervals for each signature were created using RevMan 5.3 [36]. Each entry in the forest plot represents a signature that was evaluated in a distinct cohort. Several signatures were tested in multiple cohorts and reported in multiple studies. In naming individual studies, we used the first author name, year of manuscript publication, and a sequential letter representing a specific cohort. Similarly, the signature naming convention was first author name, number of genes, and year of publication.

We reported the index test results as TP, FP, TN and FN. If not explicitly reported, TP, FP, TN, and FN were estimated from the reported sensitivity and specificity and total number of TB positive individuals and controls. Similarly, sensitivity and specificity were reported, or calculated if the data were available or obtained from the authors. We also calculated the PPV and negative predictive values (NPV) at 2% pre-test probability for predictive signatures.

Signatures used for both diagnosis and prediction of TB disease are presented in separate forest plots. Each evaluation of a signature in a different population, or of different signatures in the same population, is shown as a separate entry (or entries) in the forest plot. If the same signature was reported using different models, the best performing model was included in the analysis. Similarly, if a study reported several signatures with the same number of genes in the same population, only the best performing signature was chosen.

Considerable clinical and methodological heterogeneity was anticipated due to reporting of multiple signatures in different populations. Therefore, we did not perform meta-analysis on all signatures in all studies. In order to address this heterogeneity and compare relative performance of signatures, we performed meta-analysis only for signatures that were evaluated in at least three comparable cohorts with the same control population. Meta-analysis was conducted in STATA 11, using hierarchical logistic regression. We used the bivariate random effects model to calculate summary sensitivity and specificity with the corresponding 95% CI [37], and to create summary receiver operating characteristic curves for each signature. Heterogeneity in the diagnostic or predictive performance of the signatures was assessed by visual inspection of the forest plots and the I2 statistic.

Results

Search results

Our search term returned 2,313 reports of which 27 [23–26, 38–62] satisfied all the pre-specified inclusion criteria (Fig 1). Twenty-three of the 27 studies reported exclusively on diagnostic performance of mRNA signatures for discriminating TB disease cases from controls with or without ODs, and controls with or without LTBI. Two studies reported exclusively on predictive performance for progression to TB disease [26, 55]; and two studies reported both diagnostic and predictive performance [25, 60]. A total of 35 transcriptomic signatures incorporating 1,027 genes were identified (S2 Table). Forty-two of the 1,027 genes were employed in at least three or more transcriptomic signatures and Fc gamma receptor 1A (FCGR1A) was the most frequently utilised gene (Fig 1 in S3 File)

Quality of diagnostic studies

Four studies [42, 44, 58, 60] employed a cohort design, four [23, 38, 51, 52] a cross sectional design, and the remainder employed a case-control design. All studies had a reference standard of either smear microscopy [50], Xpert/MTB RIF [41] or MTB culture; Table 1 in S3 File. One study [46] did not specify the type of reference test used. Eight [23–25, 38, 44, 45, 49, 56, 59, 60] of the 18 studies evaluated signatures in populations with ODs. Only one study [43] made reference to blinding of the index test readers. Bias in patient selection arose in most studies due to case-control design and non-reporting of sampling method, resulting in a non-representative spectrum of patients. Bias in the index test resulted from the lack of reported blinding in the index test interpretation. Bias in the reference standard was minimal since 97% (68/70) of the entries used MTB culture as a reference standard which can be considered objective (Fig 2; Figs 2 and 3 in S3 File).

[Figure omitted. See PDF.]

Fig 2. Summary of results of QUADAS-2 assessment of diagnostic studies in clinically relevant independent validation cohorts with other diseases.

https://doi.org/10.1371/journal.pone.0237574.g002

Quality of studies for prediction of progression to TB disease

All studies [25, 26, 55, 60] validated signatures in participants from prospective independent cohorts, although two studies [25, 26] were case-control studies nested in prospective cohorts (Table 2 in S3 File). Participants from three studies did not all receive the same reference standard [26, 55, 60] and blinding was only stated in two studies [25, 26] (Fig 3; Fig 4 in S3 File).

[Figure omitted. See PDF.]

Fig 3. Summary of results of QUADAS-2 assessment in studies of prediction to TB disease in independent validation cohorts.

https://doi.org/10.1371/journal.pone.0237574.g003

Performance of mRNA signatures for TB Diagnosis

Eighteen studies evaluated 24 transcriptomic signatures for diagnosis of TB disease in 70 different independent validation cohorts (Fig 5 in S3 File). Individual signatures displayed substantial variation in diagnostic performance with sensitivity ranging from 50%-100% and specificity ranging from 32%-100%. The observed heterogeneity (I2 = 0.99) in study design and signature performance precluded pooling of diagnostic accuracy estimates for all signatures. Nine signatures were not evaluated in independent validation cohorts and hence excluded (Fig 6 in S3 File).

Thirty-three entries (46.5%) representing 17 different signatures in 12 studies met at least one TPP minimum performance criterion in independent validation sets containing uninfected controls, LTBI, ODs, or a combination of these populations (Fig 4). Signature performance ranged between 69%-100% for sensitivity, and between 70%-100% for specificity.

[Figure omitted. See PDF.]

Fig 4. Forest plots of sensitivity and specificity of transcriptomic signatures for diagnosis of TB disease that met at least one minimum TPP performance criterion in independent validation cohorts.

HC; Healthy controls, LTBI; Latent TB infection, OD; Other diseases. Vertical dashed lines correspond to 90% sensitivity and 70% specificity.

https://doi.org/10.1371/journal.pone.0237574.g004

Ten studies evaluated 12 signatures for diagnosis of TB disease in clinically relevant populations with ODs (Fig 5). Signature performance ranged between 50%-100% for sensitivity, and between 47%-96% for specificity. Seven of these signatures; Berry86_2010, daCosta2_2015, daCosta3_2015, Francisco2_2017, Kaforou44_2015, Walter47_2016, and Zak16_2016 met the WHO-recommended minimal TPP for a triage test. None of these signatures met the WHO minimal TPP for a diagnostic test in the OD population.

[Figure omitted. See PDF.]

Fig 5. Forest plots of sensitivity and specificity of transcriptomic signatures for diagnosis of TB disease in independent validation cohorts of clinically relevant populations with other diseases.

Vertical dashed lines correspond to 90% sensitivity and 70% specificity.

https://doi.org/10.1371/journal.pone.0237574.g005

Meta-analysis of performance of mRNA signatures for diagnosis of TB disease

Ten studies included seven signatures that were validated for diagnosis of TB in at least three comparable cohorts including either LTBI or OD populations. Signature diagnostic performance results are shown in Table 1. We excluded signatures validated exclusively in populations with uninfected controls, but included signatures evaluated in LTBI or OD populations that also included uninfected controls. Four signatures were validated in exclusively LTBI populations and six signatures were validated exclusively in OD populations; and two signatures in both LTBI and OD populations. Among the six signatures validated in LTBI cohorts, three signatures with similar diagnostic accuracy (Berry393_2010, Kaforou27_2013, Zak16_2016) showed pooled sensitivity and specificity that met the minimal WHO TPP for a non-sputum biomarker triage test in this population. Summary receiver operating characteristic curves for the signatures in OD populations are shown as Figs 7a, 7b and 7c in S3 File. Heterogeneity in the signature performance was not explained by comparison group (I2 = 0.98 for all OD, I2 = 0.95 for all LTBI, I2 = 0.90 for all healthy controls,). Zak16_2016 which used the TPP bench-mark of 90% sensitivity had a lower I2 of 0.56 compared to Sweeny3_2016’s I2 of 0.93. Some of the heterogeneity was explained by whether the evaluation used the TPP benchmarks and by composition of the control population.

[Figure omitted. See PDF.]

Table 1. Meta-analysis of performance of transcriptomic signatures for diagnosis of TB disease in independent validation cohorts that included LTBI or other disease populations.

https://doi.org/10.1371/journal.pone.0237574.t001

Performance of mRNA signatures for prediction of progression to TB disease

Four studies evaluated five signatures for prediction of progression to TB disease in independent validation cohorts with LTBI and uninfected controls. Two of these studies evaluated LTBI populations only; one study evaluated TB contacts only; and one study evaluated both TB contacts and LTBI populations (Fig 6). The time window between signature measurement and TB diagnosis reported in these studies was not consistent and ranged from 6 months to 24 months before TB diagnosis. Since differential gene expression becomes more pronounced as individuals approach TB diagnosis [21, 25, 26], this variable precluded direct comparison of signature performance for prediction of progression to TB disease. In LTBI populations, signature performance ranged between 76%-86% for sensitivity; and between 55%-84% for specificity. Signatures evaluated in TB contacts showed between 53%-67% sensitivity and 83%-99% specificity. The Sweeney3_2016 signature met the TPP performance criterion (PPV ≥ 5.8% and 75% sensitivity and 75% specificity) for a test to predict progression when measured within 6 months of TB diagnosis. Performance of the other signatures was not reported for this 6-month predictive interval (time to TB disease) and none of the other signatures met the TPP performance criterion over longer predictive intervals. Regardless, two other signatures, assessed within 12 months of TB diagnosis, also achieved a PPV ≥ 5.8% (Table 3 in S3 File).

[Figure omitted. See PDF.]

Fig 6. Forest plots of sensitivity and specificity of transcriptomic signatures for prediction of progression to TB disease in independent validation cohorts.

Vertical dashed lines correspond to 75% sensitivity and 75% specificity. Prediction time is the time to TB disease used in each study.

https://doi.org/10.1371/journal.pone.0237574.g006

GRADE evidence profile

We followed a previously published GRADE guideline [35] to assess the quality of the body of evidence and produced a GRADE evidence profile (Fig 7 and Fig 8 in S3 File). The overall quality of evidence supporting the estimates of sensitivity and specificity of mRNA signatures for the diagnosis of TB disease was rated as “very low”. Consequently, very low confidence is placed in the estimates obtained from pooling studies in meta-analysis. Similarly, the quality of evidence for studies of progression to TB disease was also very low.

[Figure omitted. See PDF.]

Fig 7. GRADE evidence profile: Transcriptomic signatures for the diagnosis of TB in clinically relevant populations with other diseases only.

https://doi.org/10.1371/journal.pone.0237574.g007

Discussion

The New Diagnostics Working Group Strategic Framework 2016–2020 aims to “achieve early and universal diagnosis of all patients with all forms of TB to foster progress towards TB elimination, by making appropriate and affordable diagnostic solutions available at the right setting” [63]. Comparison of signature performance in similar populations and under similar conditions is critical to down-select candidate biomarkers for development as rapid point-of-care (POC) tests. Future implementation decisions hinge on test performance in clinically relevant populations under field conditions, in addition to feasibility and cost considerations that contribute to the assessment of impact and public health value.

We show that of the 17 mRNA signatures that met at least one minimal TPP performance criterion, 11 were validated in populations including either uninfected controls, LTBI, or both, and four were validated exclusively in populations with ODs. Four signatures validated in the uninfected control category (Dewany251_2015, Sweeney3_2016, Zak16_2016, and Lee2_2016) have potential as diagnostic tests while the remainder show potential as triage tests. The signatures in the LTBI/uninfected control category that are yet to be validated in populations with ODs such as DeAraujo1_2016, Lee2_2016, Lee3_2016, and Sambarey10_2017 should also be validated in such populations to confirm robustness of diagnostic accuracy and allow comparison with the signatures above.

Although TB triage tests might be used in community mass screening campaigns or contact investigations that include uninfected individuals or asymptomatic individuals with LTBI; the greatest need is for TB diagnostic tests that discriminate TB from ODs among symptomatic individuals seeking health care. We also recognise that evaluation of novel biomarkers in cohorts of carefully selected TB cases and uninfected controls tends to over-estimate performance. In this regard, diagnostic evaluation of signatures in populations that exclusively included LTBI or uninfected controls would be considered inferior to validation of performance in clinically relevant cohorts that included individuals with other respiratory diseases. The OD category in several of these studies did not include symptomatic individuals presenting to healthcare facilities with suspected TB, but rather individuals with systemic lupus erythematosus, sarcoidosis, or other infrequently encountered conditions. None of the signatures validated in cohorts with ODs met the minimum WHO TPP for sensitivity and specificity of a diagnostic test. However, seven signatures; Berry86_2010, daCosta2_2015, daCosta3_2015, Francisco2_2017, Kaforou44_2015, Walter47_2016, and Zak16_2016 met the minimum WHO TPP for a triage test. The findings suggest that these seven signatures could be further evaluated as TB triage tests under field conditions; and that feasibility and unit cost-effectiveness as potential rapid POC tests would be a consideration for further clinical development. Other signatures that approached but did not meet the TPP may also warrant further validation in side-by-side comparison studies. Signatures with smaller number of genes which may be more adaptable to a POC device could be given preferential consideration for clinical validation and development. Our findings also suggest that further improvement in performance of existing signatures, or even further discovery of new signatures with improved performance, would be necessary for clinical development of a non-sputum TB diagnostic test. For instance, incorporating covariates such as age and sex in the models.

Validation of diagnostic tests in several geographically different populations is important to confirm robustness. Relatively few signatures were validated in at least three comparable cohorts of a similar population and were eligible for meta-analysis. Only one of the signatures (Zak16_2016), validated in multiple cohorts that included ODs such as pneumonia, lung cancer, sarcoidosis, or systemic lupus erythematosus, met the minimum WHO TPP performance criteria for a triage test in meta-analysis and none met the minimal TPP for a diagnostic test in the meta-analyses. The finding suggests that this signature has potential as a rapid POC test and should be considered for clinical development upon validation under field conditions. Similarly, other signatures approaching the minimum TPP target in meta-analysis should be considered for field validation.

Tests that will accurately predict which individuals with LTBI will develop TB disease are needed to ensure that preventive treatment can be targeted for those individuals at increased risk of incident TB disease, while saving those individuals at lowest risk from the cost, burden, and side-effects of unnecessary intervention. Current tests for MTB infection, including IGRA and TSTs, are poor predictors because of their low specificity for incident TB disease. It is not cost-effective to treat the estimated two billion individuals latently infected with MTB worldwide and therefore preventive therapy targeted with a more specific biomarker may be a more feasible alternative [64]. Predictive signatures might be used in community-level TB mass-screening campaigns or contact investigations, but might also be useful in symptomatic individuals who have been investigated and found not to have active TB disease at the time of testing. Only one of the four signatures (Sweeney3_2016) met the minimum TPP criteria for both PPV ≥ 5.8% and 75% sensitivity and 75% specificity for a 6-month period prior to TB diagnosis, although this was the only signature for which performance within six-months of TB diagnosis was reported. The other studies reported signature performance one or two years prior to TB diagnosis. It is thus not clear how these signatures would perform during the six-month period before TB diagnosis. These results highlight that more studies of predictive performance are necessary. It is also evident that a two-year predictive horizon may be overly optimistic for prediction of progression to TB disease as progression is very variable in occurrence.

One of the challenges of this systematic review and meta-analysis was that most studies did not specify whether the signature being tested was intended for triage, diagnostic, or predictive use, neither did they benchmark performance of signatures against the WHO TPPs. For example, if the goal is to discover and validate a TB triage test, a study should report specificity at 90% sensitivity or higher, to allow comparison with other novel biomarkers against this standard. Similarly, if the goal is to discover and validate a TB diagnostic test, the study should ideally report sensitivity at 98% specificity or higher; and studies aiming to discover and validate a test to predict progression to TB disease should report sensitivity at 75% specificity or higher, or PPV and NPV along with the prediction time horizon. However, it must be noted that the differences in the benchmarks are partially because some of the studies were completed before publication of the TPPs. Secondly, we found that many studies were sub-optimally designed and used control populations without clinical relevance. As observed in a previous systematic review [28], we found that several of transcriptomic signatures for TB diagnosis were discovered but not validated in independent representative cohorts. We also found that a number of diagnostic accuracy studies did not conform to the reporting guidelines for diagnostic test accuracy (DTA) studies stipulated in the “Standards for the Reporting of Diagnostic Accuracy Studies” (STARD) [65]. In several studies, cardinal data on study design, patient selection, numbers of participants in each group, and diagnostic performance data such as sensitivity and specificity with their corresponding confidence intervals (CIs) that would enable reproduction of the study were not reported. This is a major drawback in synthesising the body of evidence on DTA studies and thus compliance to STARD in designing DTA studies and reporting their findings cannot be over-emphasised.

Strengths and limitations of the study

We used an inclusive time frame of January 2005 to May 2019 to include the period in which we believe all transcriptomic TB biomarker studies were published. We also developed a protocol prior to performing the systematic review that explicitly stated a rigorous search strategy and clear inclusion/exclusion criteria. Unlike previous systematic reviews, our review includes evaluation of signatures for predicting progression to TB disease and a meta-analysis.

Some signatures were designed to optimise sensitivity while others were designed to optimise specificity. This may have introduced bias in the pooled estimates of sensitivity and specificity in the meta-analysis, and difficult to compare signature performance. Restricting included studies to those conducted in HIV-negative adults and adolescents may have excluded signatures with superior diagnostic performance in studies conducted in children or in HIV-positive individuals. Additionally, language selection bias cannot be ruled out since we only included studies reported in English. We did not formally assess publication bias as current methods are not suitable for DTA studies [66].

Heterogeneity of study design and reliance on reported data makes it impossible to fairly compare signature performance. A major finding of this study and limitation is the very low quality of evidence: preponderance of case control studies, spectrum bias and narrow geography. This highlights the need for high quality, prospective studies, with relevant populations of symptomatic clinic attendees, mass screening endemic community population or high-risk populations such as household TB contacts which minimise spectrum bias, and from multiple geographies.

Conclusion

Host blood mRNA signatures show considerable promise as triage tests for TB. Signatures designed for TB diagnosis meeting at least one TPP minimum performance criterion in independent validation sets containing healthy controls or LTBI populations should be further optimised in populations with ODs. Similarly signatures for TB diagnosis validated in populations with ODs and signatures for prediction of progression from LTBI to TB disease meeting the minimum TPP should be further optimised and validated under field conditions to confirm their accuracy for use as standalone diagnostic or predictive tests for therapeutic decision-making. There is also need for signature discovery in large “real-world” clinically appropriate populations, without spectrum bias, need for head-to-head comparison of signatures and adaptation and implementation towards a POC test.

Supporting information

[Figure omitted. See PDF.]

S1 Table. PRISMA checklist.

Checklist according to the PRISMA reporting guidelines.

https://doi.org/10.1371/journal.pone.0237574.s001

(DOCX)

S2 Table. Gene matrix.

A matrix of all identified signatures with their corresponding gene composition.

https://doi.org/10.1371/journal.pone.0237574.s002

(XLSX)

S1 File. Data extraction form.

Form used for data extraction.

https://doi.org/10.1371/journal.pone.0237574.s003

(PDF)

S2 File. QUADAS-2 form.

Modified individual study quality assessment form.

https://doi.org/10.1371/journal.pone.0237574.s004

(PDF)

S3 File. Methods and results.

Manuscript-specific supplementary methods, figures and tables.

https://doi.org/10.1371/journal.pone.0237574.s005

(PDF)

Acknowledgments

The authors acknowledge the contribution of Mrs. Mary Shelton, the Health Sciences’ reference librarian, at the University of Cape Town, for her assistance in developing the search strategies.

Citation: Mulenga H, Zauchenberger C-Z, Bunyasi EW, Mbandi SK, Mendelsohn SC, Kagina B, et al. (2020) Performance of diagnostic and predictive host blood transcriptomic signatures for Tuberculosis disease: A systematic review and meta-analysis. PLoS ONE 15(8): e0237574. https://doi.org/10.1371/journal.pone.0237574

References

1. WHO. The End TB Strategy. Geneva, Switzerland: World Health Organization; 2015.

2. WHO. Meeting Report: High-priority target product profiles for new tuberculosis diagnostics: report of a consensus meeting. Geneva, Switzerland: World Health Organization; 2014. Contract No.: WHO/HTM/TB/2014.18.

3. Seshadri P, Denkinger C. Target Product Profile: Test for Incipient Tuberculosis. Geneva, Switzerland: Foundation for Innovative New Diagnostics; 2016. Contract No.: Vs8.2; 20 October 2016.

4. WHO. Consensus Meeting Report: Development of a Target Product Profile (TPP) and a framework for evaluation for a test for predicting progression from tuberculosis infection to active disease. Report. Geneva: World Health Organization; 2017. Contract No.: WHO/HTM/TB/2017.18.

5. Ghodbane R, Raoult D, Drancourt M. Dramatic reduction of culture time of Mycobacterium tuberculosis. Scientific reports. 2014;4:4236. pmid:24577292

6. Norbis L, Miotto P, Alagna R, Cirillo DM. Tuberculosis: lights and shadows in the current diagnostic landscape. New Microbiol. 2013;36(2):111–20. pmid:23686117

7. Davies PD, Pai M. The diagnosis and misdiagnosis of tuberculosis. The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease. 2008;12(11):1226–34.

8. Steingart KR, Henry M, Ng V, Hopewell PC, Ramsay A, Cunningham J, et al. Fluorescence versus conventional sputum smear microscopy for tuberculosis: a systematic review. The Lancet Infectious diseases. 2006;6(9):570–81. pmid:16931408

9. Harries AD, Maher D, Nunn P. An approach to the problems of diagnosing and treating adult smear-negative pulmonary tuberculosis in high-HIV-prevalence settings in sub-Saharan Africa. Bull World Health Organ. 1998;76(6):651–62. pmid:10191561

10. Li S, Liu B, Peng M, Chen M, Yin W, Tang H, et al. Diagnostic accuracy of Xpert MTB/RIF for tuberculosis detection in different regions with different endemic burden: A systematic review and meta-analysis. PloS one. 2017;12(7):e0180725. pmid:28708844

11. Dorman SE, Schumacher SG, Alland D, Nabeta P, Armstrong DT, King B, et al. Xpert MTB/RIF Ultra for detection of Mycobacterium tuberculosis and rifampicin resistance: a prospective multicentre diagnostic accuracy study. The Lancet Infectious diseases. 2018;18(1):76–84. pmid:29198911

12. Maertzdorf J, Kaufmann SH, Weiner J 3rd. Toward a unified biosignature for tuberculosis. Cold Spring Harbor perspectives in medicine. 2014;5(1):a018531. pmid:25342061

13. Andrews JR, Hatherill M, Mohamed H, Hanekom WA, Campo M, Hawn TR, et al. The Dynamics of QuantiFERON-TB Gold In-Tube Conversion and Reversion in a Cohort of South African Adolescents. American journal of respiratory and critical care medicine. 2015;191(5):584–91. pmid:25562578

14. Mahomed H, Hawkridge T, Verver S, Abrahams D, Geiter L, Hatherill M, et al. The tuberculin skin test versus QuantiFERON TB Gold(R) in predicting tuberculosis disease in an adolescent cohort study in South Africa. PloS one. 2011;6(3):e17984. pmid:21479236

15. Andrews JR, Noubary F, Walensky RP, Cerda R, Losina E, Horsburgh CR. Risk of progression to active tuberculosis following reinfection with Mycobacterium tuberculosis. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2012;54(6):784–91.

16. Barry CE, 3rd, Boshoff HI, Dartois V, Dick T, Ehrt S, Flynn J, et al. The spectrum of latent tuberculosis: rethinking the biology and intervention strategies. Nature reviews Microbiology. 2009;7(12):845–55. pmid:19855401

17. Diel R, Loddenkemper R, Nienhaus A. Predictive value of interferon-gamma release assays and tuberculin skin testing for progression from latent TB infection to disease state: a meta-analysis. Chest. 2012;142(1):63–75. pmid:22490872

18. Dye C, Glaziou P, Floyd K, Raviglione M. Prospects for tuberculosis elimination. Annu Rev Public Health. 2013;34:271–86. pmid:23244049

19. Sumner T, Scriba TJ, Penn-Nicholson A, Hatherill M, White RG. Potential population level impact on tuberculosis incidence of using an mRNA expression signature correlate-of-risk test to target tuberculosis preventive therapy. Scientific reports. 2019;9.

20. Burel JG, Peters B. Discovering transcriptional signatures of disease for diagnosis versus mechanism. Nature reviews Immunology. 2018;18(5):289–90. pmid:29629714

21. Scriba TJ, Penn-Nicholson A, Shankar S, Hraha T, Thompson EG, Sterling D, et al. Sequential inflammatory processes define human progression from M. tuberculosis infection to tuberculosis disease. PLoS pathogens. 2017;13(11):e1006687. pmid:29145483

22. Blankley S, Berry MPR, Graham CM, Bloom CI, Lipman M, O'Garra A. The application of transcriptional blood signatures to enhance our understanding of the host response to infection: the example of tuberculosis. Philos T R Soc B. 2014;369(1645).

23. Berry MPR, Graham CM, McNab FW, Xu ZH, Bloch SAA, Oni T, et al. An interferon-inducible neutrophil-driven blood transcriptional signature in human tuberculosis. Nature. 2010;466(7309):973–U98. pmid:20725040

24. Kaforou M, Wright VJ, Oni T, French N, Anderson ST, Bangani N, et al. Detection of tuberculosis in HIV-infected and -uninfected African adults using whole blood RNA expression signatures: a case-control study. PLoS medicine. 2013;10(10):e1001538. pmid:24167453

25. Zak DE, Penn-Nicholson A, Scriba TJ, Thompson E, Suliman S, Amon LM, et al. A blood RNA signature for tuberculosis disease risk: a prospective cohort study. Lancet (London, England). 2016;387(10035):2312–22.

26. Suliman S, Thompson EG, Sutherland J, Weiner J, Ota MOC, Shankar S, et al. Four-Gene Pan-African Blood Signature Predicts Progression to Tuberculosis. American journal of respiratory and critical care medicine. 2018;197(9):1198–208. pmid:29624071

27. Togun TO, MacLean E, Kampmann B, Pai M. Biomarkers for diagnosis of childhood tuberculosis: A systematic review. PloS one. 2018;13(9):1–19.

28. MacLean E, Broger T, Yerlikaya S, Fernandez-Carballo BL, Pai M, Denkinger CM. A systematic review of biomarkers to detect active tuberculosis. Nat Microbiol. 2019;4(5):748–58. pmid:30804546

29. Warsinske H, Vashisht R, Khatri P. Host-response-based gene signatures for tuberculosis diagnosis: A systematic comparison of 16 signatures. PLoS medicine. 2019;16(4):e1002786. pmid:31013272

30. Gupta RK, Turner CT, Venturini C, Esmail H, Rangaka MX, Copas A, et al. Concise whole blood transcriptional signatures for incipient tuberculosis: a systematic review and patient-level pooled meta-analysis. The Lancet Respiratory medicine. 2020;8(4):395–406. pmid:31958400

31. Cochrane_Collaboration. Handbook for DTA Reviews: Cochrane; 2017 [Cochrane DTA Working Group]. Available from: http://methods.cochrane.org/sdt/handbook-dta-reviews.

32. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. pmid:25554246

33. Mulenga H, Bunyasi EW, Mbandi SK, Mendelsohn SC, Kagina B, Penn-Nicholson A, et al. Performance of host blood transcriptomic signatures for diagnosing and predicting progression to tuberculosis disease in HIV-negative adults and adolescents: a systematic review protocol. BMJ Open. 2019;9(5):e026612. pmid:31122978

34. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Annals of internal medicine. 2011;155(8):529–U104. pmid:22007046

35. Horvath AR. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. Clinical chemistry. 2009;55(5):853–5. pmid:19299540

36. Review Manager (RevMan) [Computer program]. Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.

37. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982–90. pmid:16168343

38. Bloom CI, Graham CM, Berry MP, Rozakeas F, Redford PS, Wang Y, et al. Transcriptional blood signatures distinguish pulmonary tuberculosis, pulmonary sarcoidosis, pneumonias and lung cancers. PloS one. 2013;8(8):e70630. pmid:23940611

39. Satproedprai N, Wichukchinda N, Suphankong S, Inunchot W, Kuntima T, Kumpeerasart S, et al. Diagnostic value of blood gene expression signatures in active tuberculosis in Thais: a pilot study. Genes and immunity. 2015;16(4):253–60. pmid:25764116

40. Cai Y, Yang Q, Tang Y, Zhang M, Liu H, Zhang G, et al. Increased complement C1q level marks active disease in human tuberculosis. PloS one. 2014;9(3):e92340. pmid:24647646

41. Darboe F, Mbandi SK, Thompson EG, Fisher M, Rodo M, van Rooyen M, et al. Diagnostic performance of an optimized transcriptomic signature of risk of tuberculosis in cryopreserved peripheral blood mononuclear cells. Tuberculosis. 2018;108:124–6. pmid:29523312

42. Dawany N, Showe LC, Kossenkov AV, Chang C, Ive P, Conradie F, et al. Identification of a 251 gene expression signature that can accurately detect M. tuberculosis in patients with and without HIV co-infection. PloS one. 2014;9(2):e89925. pmid:24587128

43. de Araujo LS, Vaas LA, Ribeiro-Alves M, Geffers R, Mello FC, de Almeida AS, et al. Transcriptomic Biomarkers for Tuberculosis: Evaluation of DOCK9. EPHA4, and NPC2 mRNA Expression in Peripheral Blood. Frontiers in microbiology. 2016;7:1586. pmid:27826286

44. Francisco NM, Fang YM, Ding L, Feng S, Yang Y, Wu M, et al. Diagnostic accuracy of a selected signature gene set that discriminates active pulmonary tuberculosis and other pulmonary diseases. The Journal of infection. 2017;75(6):499–510. pmid:28941629

45. Hai-Hui H, Xiao-Ying L, Yong L, Hua C, Liang-Yong X. Identification of 13 blood-based gene expression signatures to accurately distinguish tuberculosis from other pulmonary diseases and healthy controls. Bio-Medical Materials & Engineering. 2015;26:S1837–S43.

46. Huang ZK, Yao FY, Xu JQ, Deng Z, Su RG, Peng YP, et al. Microarray Expression Profile of Circular RNAs in Peripheral Blood Mononuclear Cells from Active Tuberculosis Patients. Cellular physiology and biochemistry: international journal of experimental cellular physiology, biochemistry, and pharmacology. 2018;45(3):1230–40.

47. Jacobsen M, Repsilber D, Gutschmidt A, Neher A, Feldmann K, Mollenkopf HJ, et al. Candidate biomarkers for discrimination between infection and disease caused by Mycobacterium tuberculosis. Journal of Molecular Medicine-Jmm. 2007;85(6):613–21.

48. Joosten SA, Goeman JJ, Sutherland JS, Opmeer L, de Boer KG, Jacobsen M, et al. Identification of biomarkers for tuberculosis disease using a novel dual-color RT-MLPA assay. Genes and immunity. 2012;13(1):71–82. pmid:21956656

49. Laux da Costa L, Delcroix M, Dalla Costa ER, Prestes IV, Milano M, Francis SS, et al. A real-time PCR signature to discriminate between tuberculosis and other pulmonary diseases. Tuberculosis (Edinburgh, Scotland). 2015;95(4):421–5.

50. Lee SW, Wu LS, Huang GM, Huang KY, Lee TY, Weng JT. Gene expression profiling identifies candidate biomarkers for active and latent tuberculosis. BMC bioinformatics. 2016;17 Suppl 1:3.

51. Leong S, Zhao Y, Joseph NM, Hochberg NS, Sarkar S, Pleskunas J, et al. Existing blood transcriptional classifiers accurately discriminate active tuberculosis from latent infection in individuals from south India. Tuberculosis. 2018;109:41–51. pmid:29559120

52. Lu C, Wu J, Wang H, Wang S, Diao N, Wang F, et al. Novel biomarkers distinguishing active tuberculosis from latent infection identified by gene expression profile of peripheral blood mononuclear cells. PloS one. 2011;6(8):e24290. pmid:21904626

53. Maertzdorf J, Repsilber D, Parida SK, Stanley K, Roberts T, Black G, et al. Human gene expression profiles of susceptibility and resistance in tuberculosis. Genes and immunity. 2011;12(1):15–22. pmid:20861863

54. Pan L, Wei N, Jia H, Gao M, Chen X, Wei R, et al. Genome-wide transcriptional profiling identifies potential signatures in discriminating active tuberculosis from latent infection. Oncotarget. 2017;8(68):112907–16. pmid:29348876

55. Roe J, Venturini C, Gupta RK, Gurry C, Chain BM, Sun Y, et al. Blood transcriptomic stratification of short-term risk in contacts of tuberculosis. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2019.

56. Sambarey A, Devaprasad A, Mohan A, Ahmed A, Nayak S, Swaminathan S, et al. Unbiased Identification of Blood-based Biomarkers for Pulmonary Tuberculosis by Modeling and Mining Molecular Interaction Networks. EBioMedicine. 2017;15:112–26. pmid:28065665

57. Serrano CJ, Cuevas-Cordoba B, Macias-Segura N, Gonzalez-Curiel RA, Martinez-Balderas VY, Enciso-Moreno L, et al. Transcriptional profiles discriminate patients with pulmonary tuberculosis from non-tuberculous individuals depending on the presence of non-insulin diabetes mellitus. Clinical immunology (Orlando, Fla). 2016;162:107–17.

58. Sweeney TE, Braviak L, Tato CM, Khatri P. Genome-wide expression for diagnosis of pulmonary tuberculosis: a multicohort analysis. The Lancet Respiratory medicine. 2016;4(3):213–24. pmid:26907218

59. Walter ND, Miller MA, Vasquez J, Weiner M, Chapman A, Engle M, et al. Blood Transcriptional Biomarkers for Active Tuberculosis among Patients in the United States: a Case-Control Study with Systematic Cross-Classifier Evaluation. Journal of clinical microbiology. 2016;54(2):274–82. pmid:26582831

60. Warsinske HC, Rao AM, Moreira FMF, Santos PCP, Liu AB, Scott M, et al. Assessment of Validity of a Blood-Based 3-Gene Signature Score for Progression and Diagnosis of Tuberculosis, Disease Severity, and Treatment Response. JAMA Netw Open. 2018;1(6):e183779. pmid:30646264

61. Wu B, Huang C, Kato-Maeda M, Hopewell PC, Daley CL, Krensky AM, et al. Messenger RNA expression of IL-8, FOXP3, and IL-12beta differentiates latent tuberculosis infection from disease. Journal of immunology (Baltimore, Md: 1950). 2007;178(6):3688–94.

62. Lesho E, Forestiero FJ, Hirata MH, Hirata RD, Cecon L, Melo FF, et al. Transcriptional responses of host peripheral blood cells to tuberculosis infection. Tuberculosis (Edinburgh, Scotland). 2011;91(5):390–9.

63. New Diagnostics Working Group (NDWG). The Roadmap to new TB diagnostics: Stop TB Partnership; [Available from: http://www.stoptb.org/wg/new_diagnostics/objectives.asp.

64. Sumner T, Bozzani F, Mudzengi D, Hippner P, Houben RM, Cardenas V, et al. Estimating the Impact of Tuberculosis Case Detection in Constrained Health Systems: An Example of Case-Finding in South Africa. Am J Epidemiol. 2019;188(6):1155–64. pmid:30824911

65. Bossuyt PM, Cohen JF, Gatsonis CA, Korevaar DA, Grp S. STARD 2015: updated reporting guidelines for all diagnostic accuracy studies. Ann Transl Med. 2016;4(4).

66. Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Chapter 10 Analysing and Presenting Results-Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0.: The Cochrane Collaboration; 2010 [Available from: http://srdta.cochrane.org/.