No Gold Standard Estimation of the Sensitivity and Specificity of Two Molecular Diagnostic Protocols for Trypanosoma brucei spp. in Western Kenya

Abstract

African animal trypanosomiasis is caused by a range of tsetse transmitted protozoan parasites includingTrypanosoma vivax, Trypanosoma congolense and Trypansoma brucei. In Western Kenya and other parts of East Africa two subspecies of T. brucei, T.b. brucei and the zoonoticT.b. rhodesiense, co-circulate in livestock. A range of polymerase chain reactions (PCR) have been developed as important molecular diagnostic tools for epidemiological investigations of T. brucei s.l. in the animal reservoir and of its zoonotic potential. Quantification of the relative performance of different diagnostic PCRs is essential to ensure comparability of studies. This paper describes an evaluation of two diagnostic test systems for T. brucei using a T. brucei s.l. specific PCR [1] and a single nested PCR targeting the Internal Transcribed Spacer (ITS) regions of trypanosome ribosomal DNA [2]. A Bayesian formulation of the Hui-Walter latent class model was employed to estimate their test performance in the absence of a gold standard test for detecting T.brucei s.l. infections in ear-vein blood samples from cattle, pig, sheep and goat populations in Western Kenya, stored on Whatman FTA cards. The results indicate that the system employing the T. brucei s.l. specific PCR (Se1 = 0.760) had a higher sensitivity than the ITS-PCR (Se2 = 0.640); both have high specificity (Sp1 = 0.998; Sp2 = 0.997). The true prevalences for livestock populations were estimated (pcattle = 0.091, ppigs = 0.066, pgoats = 0.005, psheep = 0.006), taking into account the uncertainties in the specificity and sensitivity of the two test systems. Implications of test performance include the required survey sample size; due to its higher sensitivity and specificity, the T. brucei s.l. specific PCR requires a consistently smaller sample size than the ITS-PCR for the detection of T. brucei s.l. However the ITS-PCR is able to simultaneously screen samples for other pathogenic trypanosomes and may thus be, overall, a better choice of test in multi-organism studies.

About the Authors:

Barend Mark de Clare Bronsvoort

* E-mail: [email protected]

Affiliation: The Roslin Institute and The Royal (Dick) School of Veterinary Studies, University of Edinburgh, Roslin, Midlothian, United Kingdom

Beatrix von Wissmann

Affiliation: Centre for Infectious Diseases, Royal (Dick) School of Veterinary Studies, University of Edinburgh, Edinburgh, United Kingdom

Eric Maurice Fèvre

Affiliation: Centre for Infectious Diseases and Centre for Infection, Immunity and Evolution, School of Biological Sciences, University of Edinburgh, Edinburgh, United Kingdom

Ian Graham Handel

Affiliation: The Roslin Institute and The Royal (Dick) School of Veterinary Studies, University of Edinburgh, Roslin, Midlothian, United Kingdom

Kim Picozzi

Affiliation: Centre for Infectious Diseases, Royal (Dick) School of Veterinary Studies, University of Edinburgh, Edinburgh, United Kingdom

Sue Christina Welburn

Affiliation: Centre for Infectious Diseases, Royal (Dick) School of Veterinary Studies, University of Edinburgh, Edinburgh, United Kingdom

Introduction

Trypanosomiasis, or ‘nagana’, is an infectious disease of livestock caused by a range of protozoan parasites. Trypanosoma vivax, Trypanosoma congolense and Trypansoma brucei are the three most important species of trypanosome, responsible for considerable production losses and livestock morbidity where they occur [3]. These parasites are transmitted by tsetse flies in the genus Glossina, in which they have obligate life cycle stages. Severity of infection with these trypanosomes depends on a range of factors; in local zebu cattle (Bos indicus) in western Kenya and elsewhere in East Africa, trypanosomiasis is an endemic disease, causing chronic anaemia [4], enlarged lymph nodes, staring coat, weakness and depression, and general loss of productivity and overall condition, including reduced milk yield and impaired fertility [5]. T. brucei, which is perhaps the least pathogenic of the three species in cattle [6], has three sub-species, namely T.b. brucei, T.b. gambiense and T.b. rhodesiense; in Western Kenya and other parts of East Africa, T.b. brucei and T.b. rhodesiense co-circulate in cattle and other livestock species. As T.b. rhodesiense is the agent of the zoonotic form of human sleeping sickness, understanding the epidemiology of T. brucei s.l. in cattle is important both for understanding and controlling animal trypanosomiasis, but also with regards to estimating the size of the reservoir of human infective parasites.

Classical estimates of sensitivity and specificity are based on direct, empirical comparisons of test outcomes for different tests, where an index test is compared to an established “gold-standard” which has an assumed sensitivity and specificity of 100%. For field diagnosis in rural settings, microscopy-based techniques using direct observation of wet blood films, or concentration techniques such as the Buffy Coat Technique, BCT [7] and the Haematocrit Centrifugation Technique, HCT [8] are the most common method of parasite detection, and have historically been considered the gold standard. Recent studies [9] have illustrated, however, that microscopy has a very poor sensitivity compared to diagnosis with molecular tools, highlighting that previous studies using these technologies are likely to have significantly underestimated both animal- and herd-level prevalence of these pathogens. This has clinical implications for the management of individual animals, but also important epidemiological implications regarding the zoonotic potential of T. brucei s.l. As a result, PCR-based diagnosis of T. brucei s.l. in livestock has now been used in a number of studies across Africa [2], [10], [11], [12], [13], using a number of different protocols and methods [1], [2], [14].

To enable comparisons between different studies, the relative performance of different testing systems needs to be quantified, preferably in such a way as to enable unbiased estimates of the true prevalence to be made, while accounting for uncertainty in the specificity and sensitivity of the system used (we refer to the testing system as the combination of the diagnostic protocol and method of sample collection and processing). We know of only one other study [15] that compared different PCR protocols (including those we examine here); while this was a valuable addition to the literature, their analysis assessed only agreement between tests and did not assess sensitivity or specificity, or indeed make estimates of the true prevalence based on the outcomes of the different tests. The development of a latent class model by Hui and Walter [16] to estimate sensitivity and specificity avoids the need for a “gold standard” which is rarely, if ever, genuinely perfect [17]. The extension of this into a Bayesian framework allows the uncertainty in the prior beliefs about the tests to be included [18] and full posterior distributions of the estimates to be given.

In the present paper, we compare two PCR-based testing systems for the detection of T. brucei s.l. in populations of cattle, pigs, sheep and goats in Western Kenya: a) a T. brucei s.l. specific primer pair [1] on material originating from ear-vein blood and stored on Whatman FTA filter cards [9]; b) ear-vein blood samples on Whatman FTA cards amplified using a single nested PCR targeting the Internal Transcribed Spacer (ITS) regions of ribosomal DNA [2]. We present estimates of sensitivity, specificity and predictive values for these two testing systems following the STARD guidelines [19] and report on the estimated true prevalence of T. brucei s.l. in livestock in two areas within the Busia District of Western Kenya.

Results

A total of 1,260 cattle, 764 goats, 311 pigs and 427 sheep were sampled across the two study sites and tested using both PCR techniques. The estimated apparent prevalence of T. brucei s.l. by each PCR method as well as the cumulative apparent prevalence are given in Table 1. The apparent prevalence is highest in cattle and pigs and lowest in sheep and goats. The estimates from each test differ only slightly, with the ITS-PCR appearing to be less sensitive than the T. brucei s.l. specific PCR. In this situation of low prevalence this difference in apparent prevalences is unlikely to biologically important. However, the different tests are clearly classifying slightly different subsets of the population as infected/uninfected, highlighted by the cumulative prevalence being higher than the individual estimates (Table 1). The species concordant and discordant test classification results are given in Table 2: these form the input for the Hui-Walter model.

[Figure omitted. See PDF.]

Table 1. The apparent prevalence estimates for each species in the study based on the individual and cumulative test results from the T. brucei s.l. specific PCR, T. brucei s.l. ITS-PCR.

https://doi.org/10.1371/journal.pone.0008628.t001

[Figure omitted. See PDF.]

Table 2. Test cross tabulation by species (T1 = T. brucei s.l. specific PCR; T2 = T. brucei s.l. ITS-PCR).

https://doi.org/10.1371/journal.pone.0008628.t002

The unbiased estimates from the Hui-Walter model are given in Table 3. The estimates of sensitivity are low for both tests but the T. brucei s.l. specific PCR on average appears to be more sensitive. Both tests are highly specific. The estimated probability distributions of the test parameters are given in the the density plots in Figure 1. These plot show the relative probability of the parameter taking a given value on the x axis and are effectively a smoothed histogram. The parameter value at the peak of the distribution represents the most likely value. The model behaved well with good mixing of the three chains as seen in the trace plots (Figure 2) which shows the samples for three chains for each parameter. They show that the chains are statistically stationary and are not autocorrelated. The Gelman-Rubin potential scale reductiom factor (PSRF) statistic for all parameters was <1.05. The PSRF is a measure of MCMC chain convergence and values substantially above 1 indicate lack of convergence [20]. Both tests have very high specificity with only moderate sensitivity, although the T. brucei s.l. specific PCR appears to be about 12% more sensitive than the ITS-PCR. The unbiased estimates of the true prevalence in each host species (Table 3) are higher for cattle and pigs than the apparent prevalence estimates for both those species, indicating that both PCR tests normally underestimate apparent prevalence. The prevalences are so low for sheep and goats that it is difficult to draw clear conclusions for these species. The estimates for sensitivity, specificity and prevelence were robust to removal of the cattle sub-population.

[Figure omitted. See PDF.]

Figure 1. Probability density plots.

Probability density plots for each test parameter estimate for the T. brucei s.l. specific PCR (sensitivity = Se[1]; specificity = Sp[1]), T. brucei s.l. ITS-PCR (sensitivity = Se[2]; specificity = Sp[2]) and the adjusted prevalence estimates from the Hui-Walter model assuming conditional independence for cattle (p[1]), goats (p[2]), pigs (p[3]) and sheep (p[4]). The x axes give the parameter estimate and the y axis the relative likelihood of it taking that value.

https://doi.org/10.1371/journal.pone.0008628.g001

[Figure omitted. See PDF.]

Figure 2. Trace plots.

MCMC history plots for each parameter of the Hui-Walter model for parameter estimates and true prevalence estimates for the T. brucei s.l. specific PCR (sensitivity = Se[1]; specificity = Sp[1]), T. brucei s.l. ITS-PCR (sensitivity = Se[2]; specificity = Sp[2]), cattle (p[1]), goats (p[2]), pigs (p[3]) and sheep (p[4]). The plots record every 10th sample from 500,000 iterations and the x axis is the sequence of iterations and the y axis the parameter value from that iteration.

https://doi.org/10.1371/journal.pone.0008628.g002

[Figure omitted. See PDF.]

Table 3. Parameter estimates (and Bayesian 95% credibility intervals, BCI) for the T. brucei s.l. specific PCR (Se1 and Sp1), T. brucei s.l. ITS-PCR (Se2 and Sp2) and adjusted T. brucei s.l. prevalence estimates for cattle (pcattle), goats (pgoats), pigs (ppigs) and sheep (psheep) from the Hui-Walter model assuming conditional independence.

https://doi.org/10.1371/journal.pone.0008628.t003

Having estimated the sensitivity and specificity for each test, we use these outputs to estimate the positive predictive values (PPV) and negative predictive values (NPV) for each test across a range of true prevalences (Figure 3). The PPV is the probability that an animal is truly positive given that it has had a positive test result. The NPV is exactly the inverse; that is, the probability that an animal is truly negative given that it is test negative. The distribution of PPVs and NPVs across a range of prevalences clearly shows that the test performance in this regard is related to prevalence - both tests have high PPVs above 20% prevalence, but this decreases rapidly as the prevalence decreases. Inversely the NPV are extremely high at lower prevalences but decrease steadily as the prevalence increases. For both the PPV and the NPV, the T. brucei s.l. specific PCR has a higher predictive value at any given prevalence than the T. brucei s.l. ITS-PCR. The estimated PPV and NPV for each test in each sub population are shown in Table 4. From this it is clear that in these sub populations, the PPV of both tests is very high for cattle and pigs, in the region of 95%, because of their relatively high prevalence but decreases markedly for sheep and goats to around 50% due to the low prevalence in these species. For the prevalences estimated in these sub-popuatlions, the NPVs were extremely high.

[Figure omitted. See PDF.]

Figure 3. Predictive value plots.

The positive (PPV) and negative predictive value (NPV) of the T. brucei s.l. specific PCR and the ITS PCR over a range of prevalence of T. brucei s.l. PPV is the positive predictive value of the test at a given prevalence i.e. how likely the animal is to have the pathogen given that it has a positive test result. NPV is the negative predictive value of the test at a given prevalence i.e. how likely is an animal not to have the pathogen given that it had a negative test result.

https://doi.org/10.1371/journal.pone.0008628.g003

[Figure omitted. See PDF.]

Table 4. Estimated predictive values of the two tests in the four sub-populations (Test 1 = T. brucei s.l. specific PCR; test 2 = T. brucei s.l. ITS-PCR).

https://doi.org/10.1371/journal.pone.0008628.t004

The impact of the test's performance on survey sample size are illustrated in Figure 4. The T. brucei s.l. specific PCR requires a consistently smaller sample size than the T. brucei s.l. ITS-PCR as expected with its higher sensitivity and specificity.

[Figure omitted. See PDF.]

Figure 4. Sample size plots.

Approximate sample sizes for a simple random survey to estimate infection prevalence using the two tests. The samples sizes are calcualted to estimate prevalence with a 95% confidence interval for an absolute precision of +/− 5%.

https://doi.org/10.1371/journal.pone.0008628.g004

Discussion

Molecular diagnostic tools, and in particular PCR, have vastly improved the detection of trypanosome infections over standard parasitological techniques, by lowering the parasitaemia detection limit by several orders of magnitude. Even when applying concentration techniques, such as the haematocrit centrifugation technique (HCT) or the buffy coat technique (BCT), the analytical sensitivity of microscopy ranges between detectable parasitaemias of 2.5×102 to 5×103 parasites/ml of blood depending on trypanosome species [21], whereas PCR can detect the presence of parasite DNA equivalent to one trypanosome in 10ml of host blood [22]. The analytic detection limit of the T. brucei s.l. specific PCR has been shown to be as low as 1/10 of the genetic material of a single trypanosome per PCR reaction [1]. The ITS - PCR was been shown to detect trypanosome DNA at a dilution equivalent to less than one parasite/ml of host blood [2]. Furthermore, primer design targeting precise DNA sequences ensures high specificity of PCR, making it independent of morphological differences required for speciation by microscopy. PCR has thus been incorporated as the diagnostic tool of choice into a wide number of studies investigating the epidemiology of trypanosomiasis, especially since advances in preservation methodology for biological samples have facilitated collection and stabilization of field samples of sufficiently high quality for molecular analysis. The field applications of PCR include estimating trypanosome prevalence for the monitoring of control programmes, though due to the cost and level of laboratory equipment involved, PCR is currently not suitable for diagnostic testing of individual animals for treatment decisions at the local level.

To enable comparison between different protocols, this study used a non-gold standard approach to quantify the parameters of the two PCR-based test systems: a T. brucei s.l. specific PCR [1] and ITS-PCR [2] run in parallel on different punches of the same wholeblood samples from cattle, pigs, goats and sheep stored on FTA-cards. This approach simultaneously allowed for an unbiased estimate of the true prevalence of T. brucei s.l. infections in the different livestock species, whilst accounting for the uncertainties in the specificity and sensitivity of the testing systems used. As expected, based on the high target-specificity of the primers designed for the respective PCRs [1], [2] the specificity of the test system was high (T.brucei s.l. specific PCR: Sp1 = 0.998; T. brucei s.l. ITS-PCR: Sp2 = 0.997) for both test systems. However, the sensitivities of the test systems were lower than expected (Se1 = 0.76, Se2 = 0.64) considering the detection limit of the PCRs themselves, which are capable of detecting very low parasitaemia. This low sensitivity may largely be due to the sample storage element of testing system - FTA cards preserve the sample by lysing cells and fixing the DNA in situ to the filter-paper matrix, and it has been shown that at low parasitaemias the parasite DNA is localized on the FTA card, with the result that using single punches for each PCR may result in negative PCR test results because the sub-sample (punch) of blood isolated for the specific reaction did not include any parasite DNA (Cox, PhD thesis 2007, University of Edinburgh). Other factors that may contribute to the low sensitivity of the testing system could include residual PCR inhibitors in the sampling material, such as haem, although the sample preparation protocol is designed to remove such known inhibitors [23]. Differences in sensitivity between the two testing systems may be attributable to a higher number of copies of the target sequence for the T. brucei s.l. specific PCR (10,000 copies/genome) as compared to the ITS-PCR (200 copies/genome) [1], [2]. It may also be that the efficiency of PCR amplifications from the FTA filter paper matrix depend on the target sequence length (1250 base pairs for the ITS-PCR and 173 for the T. brucei s.l. specific PCR), though there is no empirical evidence that supports such a negative effect.

The Hui-Walter model assumes that the sensitivity and specificity of the tests are the same across different populations. In this case we have used the different species as the different populations. For an antibody based test this would be problematic as the types of antibody response may be quite different between species. Here, however, the test is detecting the parasite DNA on an FTA card. While differences in parasitaemia between host species have been reported, differentiating a real species-specific effect from the expected individual animal level variation would be a significant undertaking [24], and we feel that it is safe for the purposes of this analysis to assume average parasite densities between host species are sufficiently similar to not affect this. Concern that co-infection of T. theileri in cattle may reduce specificity in this species was explored by re-estimation without the cattle population and estimates were found to be robust. Cross tabulation of the T. brucei tests with the T. theileri showed 27/1260 cattle and 1/427 sheep to be T. theileri positive. Only one cow was postive for both T. brucei and T. theileri with the ITS test. From these findings we find little evidence of T. brucei co-infection of T. theileri and therefore we conclude that specificity across the different sub-popualtions is unlikely to be influenced by T. theileri co-infection. The model also assumes conditional independence between the tests: i.e. given a truly positive animal, the results of the first test are independent of the second test. We believe this is justified with regards to sensitivity and specificity as the PCRs use different primer sets and target regions. The assumption that the specificity is >0.5, as reported in the methods to control the label switching issue, is justifiable as these techniques are widely accepted as highly specific because of the very nature of the technique.

The estimated true prevalences for both cattle (pcattle = 0.091) and pigs (ppigs = 0.066) were higher than those estimated from the results of the individual test systems, or indeed the cumulative prevalence derived for both tests (see Table 1), taking into account the uncertainties in the specificity and sensitivity. The under-estimation at the higher prevalences is a result of the low sensitivity and false negative results. The estimates in sheep (psheep = 0.006) and goats (pgoats = 0.005) are slightly lower than those from the cumulative test results reflecting the fact the specificity is not 1 which at very low prevalences results in low PPV for the tests and high risk of false positive results. The implications of these estimates are firstly that T. brucei s.l. is probably more widespread than currently implemented surveys based on such molecular tools suggest, both in Western Kenya [15] and elsewhere. Secondly, when designing, and assessing the impact of large scale interventions (eg [25]), the parameters of the collection and testing systems in use must be taken in to account to ensure that appropriate conclusions are drawn and recommendations made. Previous studies may have initially under-estimated the scale of the T. brucei s.l. reservoir in different livestock species, and may have under-estimated the impact that mass treatment activities have had in addressing it. Regarding the potential reservoir of zoonotic T.b. rhodesiense, cattle and pigs may be a more substantial risk than previously estimated [26] highlighting the need to specifically consider the test parameters for the T.b. rhodesiense-specific PCR protocols [14], [27] in future studies. Finally, it is imperative to standardise protocols or establish, as we do here, the relative performance of different protocols across study populations and between testing centres, in order to make meaningful comparisons between different studies. This has largely been acheived for other diseases where standard protocols with known parameters exist [28] but is lacking for a large number of non-reportable infections such as T. brucei s.l.

The T. brucei s.l. prevalence estimated from PCR results can be adjusted for the test sensitivity and specificity. However, to obtain an equal precision of estimate for the prevalence, a larger number of samples would be required when using the ITS-PCR as compared to the more sensitive T. brucei s.l. PCR. For example, at a population prevalence of 10%, 23% more samples would be required when employing ITS-PCR than when employing the specific PCR, to achieve the same absolute precision of 5% (Figure 4). Particularly for large scale studies involving several thousand individual animals, this has implications on the costs and benefits of diagnostic test choice. While our study suggests that the ITS- PCR is less sensitive than the species-specific primers, it is able to simultaneously screen samples for other pathogenic trypanosomes [2]; in multi-organism studies, it may be, overall, a better choice of test.

Methods

Ethical Statement

This study used biobank samples of blood from a number of livestock species collected from the ear vein. This non invasive approach requiring minimal restraint of the animals was approved by both the University of Edinburgh ethics review committee and the Kenyan Department of Veterinary Services.

Study Sites

The samples were collected at two study sites within Busia District, Western Province, Kenya. Site 1, located in Funyula Division, comprised nine adjacent villages. Site 2, located in Butula Division, comprised ten adjacent villages. These two sampling areas were established field sites, originally chosen on the basis of a cattle trypanosomiasis prevalence of at least 6%, as established by a cross-sectional survey in 1997 [29] and were well characterised in terms of livestock-keeping dynamics and veterinary care seeking behaviour [30], [31].

Sampling

Census sampling targeting the entire livestock population of the two sampling sites was performed in July (Funyula site) and October (Butula site) 2004, by visiting all livestock keeping homesteads in all 19 sampling villages. Whole blood samples from ear-veins were collected from all cattle (n = 1260), pigs (n = 311), goats (n = 764) and sheep (n = 427) at each livestock keeping homestead. A total of 2762 livestock samples from 549 livestock-keeping homesteads were collected. The samples (100 µl) were directly applied to FTA® Cards (Whatman, Maidstone, Kent, UK) and allowed to air dry prior to storage at room temperature, an established method of preservation for sensitive detection of trypanosome infections by PCR [9].

Laboratory Analysis

Laboratory analysis of all samples was carried out by B.v.W. in the course of her PhD, after one year of laboratory training by K.P..

Sample Preparation and PCR

All blood samples were analysed by two Polymerase Chain Reactions (PCR) according to the published protocols.For each PCR reaction one 2 mm punch was cut from the samples on the FTA ® Card and prepared according to the manufacturers instructions. Briefly, the discs were washed twice in FTA purification reagent to remove PCR inhibitors from the sample, followed by two washes with 1xTE buffer to remove residual FTA purification reagent. Once dried, the discs were transferred to PCR tubes to seed the reactions.

The first PCR targets the internal transcribed spacers (ITS) located within the ribosomal RNA genes (200 copies/genome) and detects and differentiates between the important pathogenic African trypanosome species affecting livestock, including Trypanosoma brucei s.l. [2]. The second PCR employed is specific for T. brucei s.l. with a satelite DNA target (10,000 copies/genome) [1]. One positive control (genomic DNA) and one negative control (blank FTA disc) were run with each set of reactions. PCR products were separated by electrophoresis in a 1.5% (w/v) agarose gel containing 0.5 µg/ml ethidium bromide and visualised by ultraviolet light.

Statistical Analysis

The Hui-Walter paradigm requires two (or more) tests evaluated in two (or more) populations. This model assumes that: (i) the prevalence of the disease is different within each population; (ii) the tests have the same properties across populations; and (iii) the tests are conditionally independent given the disease status. This Bayesian implementation of the Hui-Walter model [32] assumes that for the ith sub-population the counts (Oi) of the different combinations of test results, +/+, +/−, −/+ and −/− for the two tests, follow a multinomial distribution:where S is the number of subpopulations, T is the number of tests and Pri is a vector of probabilities of observing the individual combinations of test results. Conditioning on the (latent) disease status, these probabilities can be specified using the Sej and Spj of the tests and the prevalence (pi) of the sub-populations. The probabilities of observing each test combination in the ith subpopulation are given by:

In a Bayesian analysis all parameters are expressed as random variables. Prior distributions for the test properties (sensitivity and specificty) and the prevalence within the sub-populations must be specified. The sensitivity of the two tests and the prevalence in four species were given flat (Uniform(0,1) priors) as there were no published data to inform these estimates. Each test's specificity was given a uniform prior over the range 0.5–1.0. This assumption is still vague but by constraining the specificity above 0.5 we control the label switching issue of Markov chain Monte Carlo (MCMC). This issue is discussed in detail with reference to the estimation of Hui-Walters models by Toft et al. [32]. In order to explore the influence of the large catle sub-population the model was re-estimated using only the small ruminant and pig data.

The model was estimated using the JAGS software [33] using the Runjags package [34] of the R statistical system [35] Three MCMC chains were run for this analysis. The first 500,000 iterations were discarded as a burn-in and the following 500,000 iterations were kept and thinned to 50,000 for posterior inference. Convergence of the chains after the initial burn-in was assessed by visual inspection of the time-series plots for the parameter samples as well as Gelman-Rubin diagnostic plots using three sample chains with dispersed starting values [36]. The R package CODA [20] was used for analysis and graphing of the McMC output.

The impact diagnostic test performance can be illustrated by estimation of predictive values, the probability of a tested individual having a given infection status condition on its test result. The positive and predictive values are given by the following formulae:(equation 1)(equation 2)

Where Se is the estimated test sensitivity, Sp is the estimated test specificity, P is the true seroprevalence in the population.

In a classical analysis the single, point estimates of the diagnostic test's sensitivity and specificty are used in these estimators. The Bayesian implementation of the Hui-Walters model produces full, joint posterior estimates of these parameters. We estimated the predictive values of the diagnostic test results over a range of prevalences by numerically integrating equation 1 and equation 2 over the paired estimates of sensitivity and specificity from the model. This approach incorporates the uncertainty and covariance structure of the test performance into the predictive values.

This methodology was extended to estimate the positive and negative predictive values of the two tests in the four study sub-populations by numerically integrating over the joint posterior distribution of prevalence and test sensitivity and specificity for each test/population combination. It is likely that a major application of these tests will be in large scale surveys for the estimation of infection prevalence. Estimates from such surveys will be uncertain due to sampling and imperfections of the diagnostic test. Conventionally these surveys are designed to estimate infection prevalence to a required precision [37]. The required sample size in a survey is a function of the expected prevalence, the diagnostic test performance and the required precision. To estimate the impact of the different diagnostic test performances we calculate approximate sample sizes that would be required for a simple survey over a range of prevalences using the two tests (for a given precision). The analysis uses the following formula for sample size [37] to calculate approximate samples sizes for a 95% confidence with a given absolute error.

Where Se and Sp are the diagnostic test sensitivity and specificity, p is the expected prevalence and d is the absolute proportional error.

Acknowledgments

We thank the District Veterinary Officer, Busia District, Kenya for his assistance and support during sample collection in Funyula and Butula Divisions. In particular, we would like to thank the animal health assistants, George Ouma, Robert Sanya, Alex Jonyo, Agrey Nying'uro and Thomas Situma and the ILRI driver Mr Kaigai Karanja for their hard work and assistance during the study, as well as Noreen Machila and Andrew P. Cox for their help with sample collection.

Author Contributions

Conceived and designed the experiments: BMdCB BvW EMF KP SCW. Performed the experiments: BvW KP. Analyzed the data: BMdCB IGH. Wrote the paper: BMdCB BvW EMF IGH SCW.

Citation: Bronsvoort BMdC, Wissmann Bv, Fèvre EM, Handel IG, Picozzi K, Welburn SC (2010) No Gold Standard Estimation of the Sensitivity and Specificity of Two Molecular Diagnostic Protocols for Trypanosoma brucei spp. in Western Kenya. PLoS ONE5(1): e8628. https://doi.org/10.1371/journal.pone.0008628

References

1. Moser DR, Cook GA, Ochs DE, Baily CP, McKane MR, et al. (1989) Detection of Trypanosoma congolense and Trypanosoma brucei subspecies by DNA amplification using the polymerase chain reaction. Parasitology 99: 57–66.DR MoserGA CookDE OchsCP BailyMR McKane1989Detection of Trypanosoma congolense and Trypanosoma brucei subspecies by DNA amplification using the polymerase chain reaction.Parasitology995766

2. Cox A, Tilley A, McOdimba F, Fyfe J, Eisler MC, et al. (2005) A PCR based assay for detection and differentiation of African trypanosome species in blood. Experimental Parasitology 111: 24–29.A. CoxA. TilleyF. McOdimbaJ. FyfeMC Eisler2005A PCR based assay for detection and differentiation of African trypanosome species in blood.Experimental Parasitology1112429

3. Kristjanson PM, Swallow BM, Rowlands GJ, Kruska RL, de Leeuw PN (1999) Measuring the costs of African animal trypanosomosis, the potential benefits of control and returns to research. Agricultural Systems 59: 79–98.PM KristjansonBM SwallowGJ RowlandsRL KruskaPN de Leeuw1999Measuring the costs of African animal trypanosomosis, the potential benefits of control and returns to research.Agricultural Systems597998

4. Taylor K, Authié E (2004) Pathogenesis of animal trypanosomiasis. In: Maudlin I, Holmes PH, Miles MA, editors. The Trypanosomiases. Wallingford, Oxfordshire: CABI Publishing. pp. 331–353.K. TaylorE. Authié2004Pathogenesis of animal trypanosomiasis.I. MaudlinPH HolmesMA MilesThe TrypanosomiasesWallingford, OxfordshireCABI Publishing331353

5. Connor RJ (1994) The impact of nagana. Onderstepoort Journal of Veterinary Research 61: 379–383.RJ Connor1994The impact of nagana.Onderstepoort Journal of Veterinary Research61379383

6. Uilenberg G (1998) A field guide for the diagnosis, treatment and prevention of African animal trypanosomiasis. Rome: Food and Agricultural Organization of the United Nations. G. Uilenberg1998A field guide for the diagnosis, treatment and prevention of African animal trypanosomiasis.RomeFood and Agricultural Organization of the United Nations

7. Murray M, Murray PK, McIntyre WIM (1977) An improved parasitological technique for the detection of African trypanosomiasis. Transactions of the Royal Society of Tropical Medicine and Hygiene 71: 325–326.M. MurrayPK MurrayWIM McIntyre1977An improved parasitological technique for the detection of African trypanosomiasis.Transactions of the Royal Society of Tropical Medicine and Hygiene71325326

8. Woo PTK (1970) The haematocrit centrifugation technique for the diagnosis of African trypanosomiasis. Acta Tropica 27: 384–386.PTK Woo1970The haematocrit centrifugation technique for the diagnosis of African trypanosomiasis.Acta Tropica27384386

9. Picozzi K, Tilley A, Fèvre EM, Coleman PG, Magona JW, et al. (2002) The diagnosis of trypanosome infections: application of novel technology for reducing disease risk. African Journal of Biotechnology 1: 39–45.K. PicozziA. TilleyEM FèvrePG ColemanJW Magona2002The diagnosis of trypanosome infections: application of novel technology for reducing disease risk.African Journal of Biotechnology13945

10. Clausen P-H, Wiemann A, Patzel R, Kakaire D, Poetzsch C, et al. (1998) Use of a PCR assay for the specific and sensitive detection of trypanosoma spp. in naturally infected dairy cattle in peri-urban Kampala, Uganda. Annals of the New York Academy of Sciences 849: 21–31.P-H ClausenA. WiemannR. PatzelD. KakaireC. Poetzsch1998Use of a PCR assay for the specific and sensitive detection of trypanosoma spp. in naturally infected dairy cattle in peri-urban Kampala, Uganda.Annals of the New York Academy of Sciences8492131

11. Ng'ayo MO, Njiru ZK, Kenya EU, Muluvi GM, Osir EO, et al. (2005) Detection of trypanosomes in small ruminants and pigs in western Kenya: important reservoirs in the epidemiology of sleeping sickness? Kinetoplastid Biology and Disease 4: 5.MO Ng'ayoZK NjiruEU KenyaGM MuluviEO Osir2005Detection of trypanosomes in small ruminants and pigs in western Kenya: important reservoirs in the epidemiology of sleeping sickness?Kinetoplastid Biology and Disease45

12. Pinchbeck GL, Morrison LJ, Tait A, Langford J, Meehan L, et al. (2008) Trypanosomiasis in The Gambia: prevalence in working horses and donkeys detected by whole genome amplification and PCR, and evidence for interaction between trypanosome species. BMC Veterinary Research 4: GL PinchbeckLJ MorrisonA. TaitJ. LangfordL. Meehan2008Trypanosomiasis in The Gambia: prevalence in working horses and donkeys detected by whole genome amplification and PCR, and evidence for interaction between trypanosome species.BMC Veterinary Research4

13. Solano P, Michel JF, Lefrancois T, de La Rocque S, Sidibe I, et al. (1999) Polymerase chain reaction as a diagnosis tool for detecting trypanosomes in naturally infected cattle in Burkina Faso. Veterinary Parasitology 86: 95–103.P. SolanoJF MichelT. LefrancoisS. de La RocqueI. Sidibe1999Polymerase chain reaction as a diagnosis tool for detecting trypanosomes in naturally infected cattle in Burkina Faso.Veterinary Parasitology8695103

14. Picozzi K, Carrington M, Welburn SC (2008) A multiplex PCR that discriminates between Trypanosoma brucei brucei and T.b. rhodesiense. Experimental Parasitology 118: 41–46.K. PicozziM. CarringtonSC Welburn2008A multiplex PCR that discriminates between Trypanosoma brucei brucei and T.b. rhodesiense.Experimental Parasitology1184146

15. Thumbi SM, McOdimba FA, Mosi RO, Jung'a JO (2008) Comparative evaluation of three PCR base diagnostic assays for the detection of pathogenic trypanosomes in cattle blood. Parasites and Vectors 1: 46.SM ThumbiFA McOdimbaRO MosiJO Jung'a2008Comparative evaluation of three PCR base diagnostic assays for the detection of pathogenic trypanosomes in cattle blood.Parasites and Vectors146

16. Hui SL, Walter SD (1980) Estimating the error rates of diagnostic tests. Biometrics 36: 167–171.SL HuiSD Walter1980Estimating the error rates of diagnostic tests.Biometrics36167171

17. Enøe C, Georgiadis MP, Johnson WO (2000) Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. Preventive Veterinary Medicine 45: 61–81.C. EnøeMP GeorgiadisWO Johnson2000Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown.Preventive Veterinary Medicine456181

18. Bronsvoort BMd, Koterwas B, Land F, Handel IG, Tucker J, et al. (2009) Comparison of a flow assay for brucellosis antibodies with the reference cELISA test in West African Bos Indicus. PloS One 4: BMd BronsvoortB. KoterwasF. LandIG HandelJ. Tucker2009Comparison of a flow assay for brucellosis antibodies with the reference cELISA test in West African Bos Indicus.PloS One4

19. Bossuyet PM, Reitsma JB, Bruns DE, Gatsonis CA, Glaszious PP, et al. (2003) The STARD Statement for reporting studies of diagnostic accuracy: explanation and elaboration. Annals of Internal Medicine 138: W1–12.PM BossuyetJB ReitsmaDE BrunsCA GatsonisPP Glaszious2003The STARD Statement for reporting studies of diagnostic accuracy: explanation and elaboration.Annals of Internal Medicine138W112

20. Plummer M, Best N, Cowles K, Vines K (2009) Output analysis and diagnostics for MCMC. M. PlummerN. BestK. CowlesK. Vines2009Output analysis and diagnostics for MCMC.R package version 0.13-4. R package version 0.13-4.

21. Eisler MC, Dwinger RH, Majiwa PAO, Picozzi K (2004) Diagnosis and epidemiology of African animal trypanosomiasis. In: Maudlin I, Holmes PH, Miles MA, editors. The trypanosomiases. Wallingford, Oxfordshire: CABI Publishing. MC EislerRH DwingerPAO MajiwaK. Picozzi2004Diagnosis and epidemiology of African animal trypanosomiasis.I. MaudlinPH HolmesMA MilesThe trypanosomiasesWallingford, OxfordshireCABI Publishing

22. Masake RA, Njuguna JT, Brown CC, Majiwa PAO (2002) The application of PCR-ELISA to the detection of Trypanosoma brucei and T. vivax infections in livestock. Veterinary Parasitology 105: 179–189.RA MasakeJT NjugunaCC BrownPAO Majiwa2002The application of PCR-ELISA to the detection of Trypanosoma brucei and T. vivax infections in livestock.Veterinary Parasitology105179189

23. Whatman FTA Cards (Product Sheet) - Collect, archive, transport and purify nucleic acids at all room temperatures. Whatman FTA Cards (Product Sheet) - Collect, archive, transport and purify nucleic acids at all room temperatures.

24. Murray M (1989) Factors affecting duration and intensity of trypanosome infections of domestic animals. Annales de la Societe Belge de Medecine Tropicale 69: 189–196.M. Murray1989Factors affecting duration and intensity of trypanosome infections of domestic animals.Annales de la Societe Belge de Medecine Tropicale69189196

25. Kabasa JD (2007) Public-private partnership works to stamp out sleeping sickness in Uganda. Trends in Parasitology 23: 191–192.JD Kabasa2007Public-private partnership works to stamp out sleeping sickness in Uganda.Trends in Parasitology23191192

26. Waiswa C, Olaho-Mukani W, Katunguka-Rwakishaya E (2003) Domestic animals as reservoirs for sleeping sickness in three endemic foci in south-eastern Uganda. Annals of Tropical Medicine and Parasitology 97: 149–155.C. WaiswaW. Olaho-MukaniE. Katunguka-Rwakishaya2003Domestic animals as reservoirs for sleeping sickness in three endemic foci in south-eastern Uganda.Annals of Tropical Medicine and Parasitology97149155

27. Welburn SC, Picozzi K, Fèvre EM, Coleman PG, Odiit M, et al. (2001) Identification of human-infective trypanosomes in animal reservoirs of sleeping sickness in Uganda by means of serum-resistance-associated (SRA) gene. The Lancet 358: 2017–2019.SC WelburnK. PicozziEM FèvrePG ColemanM. Odiit2001Identification of human-infective trypanosomes in animal reservoirs of sleeping sickness in Uganda by means of serum-resistance-associated (SRA) gene.The Lancet35820172019

28. OIE (2009) Manual of Diagnostic Tests and Vaccines for Terrestrial Animals 2009. OIE2009Manual of Diagnostic Tests and Vaccines for Terrestrial Animals 2009.

29. INCO-DC (2000) Novel approaches to the epidemiology of resistance to drugs used in the control of bovine trypanosomiasis in East Africa. Final Report. Glasgow: International Cooperation with Developing Countries (1994–1998). INCO-DC2000Novel approaches to the epidemiology of resistance to drugs used in the control of bovine trypanosomiasis in East Africa. Final Report.GlasgowInternational Cooperation with Developing Countries (1994–1998)

30. Machila N, Wanyangu SW, McDermott J, Welburn SC, Maudlin I, et al. (2003) Cattle owners' perceptions of African bovine trypanosomiasis and its control in Busia and Kwale Districts of Kenya. Acta Tropica 86: 25–34.N. MachilaSW WanyanguJ. McDermottSC WelburnI. Maudlin2003Cattle owners' perceptions of African bovine trypanosomiasis and its control in Busia and Kwale Districts of Kenya.Acta Tropica862534

31. Thuranira CM (2005) Socio-economic factors influencing livestock keeping dynamics in a smallholder crop-livestock system in western Kenya [PhD Thesis]. Edinburgh: University of Edinburgh. 311 p.CM Thuranira2005Socio-economic factors influencing livestock keeping dynamics in a smallholder crop-livestock system in western Kenya [PhD Thesis].EdinburghUniversity of Edinburgh311

32. Toft N, Jørgensen E, Højsgaard S (2005) Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Preventive Veterinary Medicine 68: 19–33.N. ToftE. JørgensenS. Højsgaard2005Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard.Preventive Veterinary Medicine681933

33. Plummer M (2003) JAGS: A program for analysis of Bayesian Graphical Models using Gibbs Sampling. Distributed Statistical Computing. Vienna. M. Plummer2003JAGS: A program for analysis of Bayesian Graphical Models using Gibbs Sampling. Distributed Statistical Computing. Vienna.

34. Denwood (2009) Denwood2009runjags: Run Bayesian MCMC Models in the BUGS syntax from within R. Available from http://cran.r-project.org/web/packages/runjags/. runjags: Run Bayesian MCMC Models in the BUGS syntax from within R. Available from http://cran.r-project.org/web/packages/runjags/.

35. R Development Core Team (2009) R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.R Development Core Team2009R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

36. Brooks SP, Gelman A (1998) Alternative methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics 7: 434–455.SP BrooksA. Gelman1998Alternative methods for monitoring convergence of iterative simulations.Journal of Computational and Graphical Statistics7434455

37. Thrusfield M (1995) Veterinary Epidemiology. Cambridge: Blackwell Science Ltd. M. Thrusfield1995Veterinary Epidemiology.CambridgeBlackwell Science Ltd