Recent studies in several countries have shown that a high prevalence of follicular trachoma (TF) can persist for months or years in communities in which the prevalence of ocular C. trachomatis
has been reduced to a very low level, or even eliminated, following mass treatment (32
). Current recommendations are to continue mass treatment with azithromycin until the prevalence of TF in children ages 1 to 9 years is below 5%, but this can lead to repeated rounds of mass treatment for communities in which few, if any, individuals are infected. Basing the decision to stop mass treatment on the prevalence of C. trachomatis
infection would save resources and reduce the risk of selecting for macrolide-resistant strains of important pathogens, such as Streptococcus pneumoniae
). As the prevalence of infection falls, so does the positive predictive value of a test for infection. It is therefore particularly important, in designing a test that will be used to determine that infection has been eliminated from a community, to ensure that it is highly specific. Based on the evidence of a comparison to a well-validated reference test, Amplicor CT/NG, the ddPCR assay fulfills this criterion.
The standard calibration curve is a simultaneous test in which a single metric, the coefficient of determination (the r2 value), is used to describe the accuracy and intraoperator precision of PCR assays. Using a serial dilution calibration series, we found that the ddPCR test is a highly precise and accurate (r2 = ~0.9927) assay that can detect and quantify chlamydial plasmids across a wide range of concentrations. Sampling error in the ddPCR test was assessed by examination of the 95% confidence intervals () of the concentration estimates. The increase in sampling error that we observed at lower dilutions clearly reflected the stochastic sampling process that takes place when an aliquot is taken from a larger specimen volume. Slight variation in the levels of RPP30 that were detected () was able to be explained by error introduced by liquid-handling procedures.
We chose to use a confidence measure (ζ) to classify the results of ddPCR tests. This value is derived from the normal probability distribution of the ddPCR concentration and describes the probability that the true concentration is above zero copies/μl. For the classification of specimens as positive for the PCR targets of either the human or C. trachomatis assays, we initially chose an arbitrary cutoff value for ζ of 0.95. At this threshold, there is 95% confidence that the true target concentration is not zero. In using a threshold of ζ of 0.95, we define a highly specific assay in which there is a theoretical maximum probability of false-positive results in any given test that is less than 5%.
We took a standard approach to diagnostic evaluation and used ROC analysis to compare the ddPCR test results to a set of Amplicor CT/NG data that had been classified according to the most recent version of the manufacturer's protocol (rev. 3.0; 2010). In this case, we used the estimated concentration of plasmids per microliter of the PCR mixture as the classifier. This value, which is proportional to ζ, is the biologically meaningful quantitative output of the ddPCR assay. We found that ddPCR performed extremely well in this population (ROC AUC was above 90%; Amplicor CT/NG estimated prevalence was ~17.47%). shows how differing threshold values of the ddPCR classifier can be used to optimize the assay (highly sensitive/highly specific/balanced) for use in different prevalence settings.
The evidence from our direct comparison to Amplicor CT/NG indicated that the ddPCR test was less sensitive (maximally 83.7%) than Amplicor CT/NG. We acknowledge that our ddPCR C. trachomatis
test may be inferior to Amplicor CT/NG both in general terms and particularly with respect to sensitivity. The observation of lower-than-expected sensitivity is similar to the findings of a previous study (22
), in which a human cytomegalovirus (CMV) ddPCR assay was found to be less sensitive than a qPCR assay for the same target. These findings are surprising given the theoretical technological advantage of ddPCR over conventional microliter-scale fluidic PCR. If false-positive classification by the Amplicor CT/NG assay had occurred, the result would be a false indication of low sensitivity in ddPCR. Previous studies have used discrepant analysis to resolve divergent results between the index and reference tests (36
), and conditionally independent tests to arbitrate discrepancies (with proven high sensitivity), such as the Gen-Probe Aptima test, (48
) are available. This test is based on C. trachomatis
RNA detection, and we did not have access to RNA specimens or any other biological material from the individuals who participated in the study. The use of a third DNA-based test, such as qPCR, would not be conditionally independent, and the validity of arbitration by discrepant analysis remains controversial in any case (49
The diagnostic classifiers in the qualitative Amplicor CT/NG (OD450) and quantitative ddPCR (estimated target concentration) tests are both continuous variables. Although the Amplicor CT/NG OD450 value is not thought to be an accurate estimate of the DNA targets contained in the original specimen, this value is the continuous quantitative measure that is used to define boundaries for sample positivity. Changing the critical threshold value at which positive/negative classification is made can affect the outcome of comparative diagnostic evaluations, as demonstrated in . The choice of threshold in the reference is therefore a critical determinant of the diagnostic evaluation because misclassification by the reference has deleterious downstream effects on the estimation of the diagnostic performance of the comparative test. shows how a large number (n = 172) of data points fell within what we describe as an expanded equivocal zone in the Amplicor CT/NG data (0.1 to 3.1), wherein the classification would change if different Amplicor CT/NG thresholds were chosen.
In the product insert for the Amplicor CT/NG test (rev. 3.0; 2010), the manufacturer recommended a cutoff threshold for Amplicor CT of 0.8 OD450
units, with triplicate testing indicated for all equivocal values between 0.2 and 0.8 in the initial test. Earlier versions (available online at http://www.fda.gov/downloads/BiologicsBloodVaccines/SafetyAvailability/TissueSafety/ucm100246.pdf
) present figures of data and recommend a much wider equivocal zone for which testing by alternate methods would be required (0.2 to 2.0). We believe that the most recent guidelines limit the equivocal range to the area in which false positives are most likely (OD450
= 0.2 to 0.8) instead of the area in which they are possible (OD450
= 0.2 to ~3) and that this may increase the cost-effectiveness of C. trachomatis
screening, as these guidelines require fewer replicate or alternate tests than the earlier guidelines. The result of these measures is the implementation of a highly sensitive test, but with the consequence of a significant number of false-positive results. The Amplicor test guidelines were optimized for use in testing for genitourinary tract infections, for which the risk of false-negative results would be minimized. While Amplicor CT/NG has been validated and widely used in testing ocular samples (8
), any misclassification that has occurred in this study might be the result of using Amplicor classification cutoff values that are not appropriate to the ocular specimens. Nonetheless, when we chose a ddPCR threshold (~0.07 plasmids/μl) that maximized sensitivity, 42 Amplicor-positive but ddPCR-negative specimens remained. We anticipate that some proportion of these is attributable to misclassification by Amplicor CT/NG testing.
A second source of error in the sensitivity estimate is shown in . We assessed samples that were ddPCR and Amplicor CT/NG (i.e., OD450 of >0.8) positive and found that the ddPCR estimate of the plasmid concentration was uniformly low (median, 0.24 plasmids/μl) when the Amplicor CT/NG OD450 value was in the range of 0.2 to 3.1, which we defined as the “expanded equivocal zone.” Samples in the Amplicor CT-positive range (OD450 > 3.1) had much higher concentration estimates by ddPCR (median, 87.99 plasmids/μl). We take these observations to indicate that a significant number of the discrepancies are the result of sampling error (which affects the performance of most NAATs) and/or suboptimal performance when presented with a specimen that has a low target sequence load.
We developed a highly specific test that is widely applicable for C. trachomatis
testing and may be useful in monitoring infection post-mass drug administration (MDA). We aim to maintain high specificity, which we can define by a stringent threshold value of ζ of ≥0.95. Reducing the threshold value of ζ would lead to dramatic increases in sensitivity, but the confidence with which we would accept a sample as positive is reduced, and this might lead to overdiagnosis. This is particularly important when or if MDA treatment decisions are informed by a test for current ocular chlamydial infection rather than by clinical signs of trachoma alone (51
Previous studies by us and others have suggested that there is a threshold of C. trachomatis
infection prevalence and intensity below which the infection will spontaneously disappear from a population—the so-called Allee effect (5
). A test that accurately estimates the bacterial load, such as the ddPCR assay, may be especially valuable in determining when this threshold has been reached and it is therefore safe to discontinue MDA.
The ddPCR technology has great utility as a diagnostic. Unlike most other NAATs, this method requires no external or internal calibration yet delivers a highly accurate estimation of target load. The inclusion of a human target allows a system of internal control that indicates (but does not discriminate between) experimental failures that are due to either technical PCR failure or the absence of a sample of human origin in the specimen.
We have validated a highly specific quantitative ddPCR assay and applied it to population monitoring of ocular C. trachomatis infections. This method can rapidly be adapted for use in the detection of sexually transmitted C. trachomatis infections, for which a higher sensitivity than that which we have estimated would be desirable. This study represents an early example of the usefulness of this next-generation digital PCR method in diagnosing infections in clinical specimens within the arena of infectious diseases.