|Home | About | Journals | Submit | Contact Us | Français|
We evaluated three of the CDC approaches for confirming Neisseria gonorrhoeae (gonococcus [GC])-positive nucleic acid amplification test (NAAT) results: (i) repeating the original test on the original specimen, (ii) testing the original specimen with a different test, and (iii) performing a different test on a duplicate specimen collected at the same visit. For the first approach, clinical specimens were initially tested by Aptima Combo 2 (AC2) (Gen-Probe Inc., San Diego, CA), ProbeTec (strand displacement amplification [SDA]) (Becton Dickinson Co., Sparks, MD), and Amplicor (PCR) (Roche Molecular Systems, Branchburg, NJ). The original GC-positive specimens were then retested by the same NAAT for confirmation. For the second approach, specimens initially positive by AC2, SDA, or PCR were retested by different NAATs (SDA, PCR, AC2, and Aptima Neisseria gonorrhoeae assay [AGC]; Gen-Probe Inc.). For the third approach, duplicate urethral swabs and first-catch urine (FCU) samples from men and duplicate cervical swabs and FCU samples from women were each tested by SDA, AC2, and AGC in parallel. We found that 89 to 96% of samples positive by SDA, PCR, and AC2 were confirmed by repeat testing and that 85 to 98% of SDA, PCR, and AC2 results were confirmed by using different NAATs on the original specimen. For FCU samples from men, any NAAT can be used for confirmation. However, for all other specimen types, some NAATs cannot be used to confirm positive results from other NAATs. Thus, a single repeat test appears to be a reliable method for confirmation, but by doing more extensive testing, an additional 5% were confirmed. With >90% of all GC-positive NAATs being confirmed, our results show that confirmatory testing is not warranted for these genital specimens.
Nucleic acid amplification tests (NAATs) are highly sensitive and specific for the detection of Chlamydia trachomatis and Neisseria gonorrhoeae (gonococcus [GC]) using swabs from the genital tract or first-catch urine (FCU) specimens (2, 8, 11, 12, 18). These tests are widely available and are used routinely for screening men and women. In a 2004 survey, Dicker et al. reported that 78.7% of public health laboratories performed NAATs for GC (4). The majority of laboratories use either the ProbeTec (strand displacement amplification [SDA]) (Becton Dickinson Co., Sparks, MD) or the Aptima Combo 2 (AC2) (Gen-Probe Inc., San Diego, CA) assay, and fewer use the Amplicor (PCR) (Roche Diagnostics Corp., Branchburg, NJ) test. Although NAATs are commonly used, there have been some concerns about test specificity, particularly in low-prevalence populations, where the positive predictive values (PPVs) would be negatively impacted (5, 10, 13). False-positive (FP) results can occur with SDA and PCR, as their target may cross-react with several different species of Neisseria (N. cinerea, N. flavescens, N. lactamica, N. subflava, and N. sicca) (6, 19). To date, this type of FP result has not been observed with the AC2 or the Aptima Neisseria gonorrhoeae (AGC) (Gen-Probe Inc., San Diego, CA) test.
Because of potential FP results and the relatively low specificities (97.6 to 98.9%) in the GC NAAT package inserts, the Centers for Disease Control and Prevention (CDC) has recommended confirmatory testing of positive GC NAAT results when the PPV is <90% (1). In theory, this additional testing should improve test specificity.
Confirmatory testing is controversial; it results in added laboratory costs and delays patient reports. The 2002 CDC-recommended approaches for supplemental testing were implemented without being thoroughly evaluated. Only two large studies on C. trachomatis or GC NAAT result confirmation have been published (9, 14). Conclusions of both evaluations are contrary to the CDC's recommendation of confirmatory testing. Indeed, our evaluation of three CDC-suggested approaches found that C. trachomatis confirmatory testing is unnecessary (14). In low- to moderate-prevalence populations, >90% of positive C. trachomatis NAAT results were confirmed.
In a low (0.5%)-prevalence population, Golden et al. found that the routine confirmation of GC-positive specimens using the Gen-Probe AC2 assay was not necessary (9). They performed 265 confirmatory AGC tests on cervical and urine specimens positive by AC2, and 258 were positive, with a PPV of 97.4%. However, their evaluation studied only that one method of confirmation. Clearly, more evaluations are needed.
From 2003 to 2007, we performed supplemental testing on GC-positive samples tested by NAAT using the three different CDC-suggested approaches with cervical, urethral, rectal, pharyngeal, glans, and FCU specimens. In addition, we had the opportunity to analyze data from the clinical trial to evaluate the performance of the AGC test (2, 8). In this study, we present our findings with the following NAAT confirmation strategies: (i) repeating the original test on the original specimen, (ii) performing a second NAAT targeting a different nucleic acid sequence on the original specimen, and (iii) testing a second specimen with a different test or target.
We evaluated the following NAATs for GC confirmatory testing: ProbeTec (SDA) (Becton Dickinson Co., Sparks, MD), Amplicor (PCR) (Roche Diagnostics Corp., Branchburg, NJ), AC2 (Gen-Probe Inc., San Diego, CA), and AGC, another Gen-Probe test that targets rRNA sequences different from those of the AC2 test. For each NAAT, tests were performed according to instructions included in the package insert. All specimens were collected in either a universal transport medium (M4; Remel, Lenexa, KS) or NAAT-specific transport medium. This study was approved by the Committee on Human Research at the University of California, San Francisco.
Subjects attended sexually transmitted disease, obstetrics/gynecology, and family-planning clinics in California and a sexually transmitted disease clinic in Indonesia. Symptomatic and asymptomatic patients were tested. We tested endocervical swabs and FCU from women. We tested urethral swabs and FCU from men. We tested pharyngeal, rectal, and glans swabs from men who have sex with men (MSM). It should be noted that these specimens from MSM are not FDA cleared for use with NAATs. However, our laboratory validated these sample types with in-house evaluations (the pharyngeal swab was found to be unacceptable for use with PCR). All specimens were initially tested by an NAAT for N. gonorrhoeae detection. Specimens that were originally positive were saved for confirmatory testing. Specimens were either held at 4°C and retested within 48 h or stored at −70°C until confirmatory testing was performed.
From June 2003 to May 2007, clinical specimens were initially tested by AC2. All positive specimens were repeat tested by AC2 and by the Aptima GC assay, a different NAAT. In addition, positive specimens with initial low-level readings (between 200 to 1,000 relative light units [RLU]) were retested by AC2 on five separate assay runs.
From to June 2003 to March 2004, clinical specimens collected from MSM were initially tested by PCR. All positive specimens were repeat tested by PCR.
From January 2003 to May 2007, clinical specimens were initially tested by SDA. All positive specimens were repeat tested by SDA. For specimens with adequate volumes, a third repeat test and a fourth repeat test by SDA were also performed. In addition, positive specimens with initial low-level readings (between 2,000 and 15,000 mota) and some seeded samples with a dilution of N. gonorrhoeae expected to yield a low-level positive result were retested on four and seven separate assay runs, respectively.
From January 2005 to March 2005, clinical specimens were tested by AC2. Specimens with an initial GC-negative result were identified. A random selection of 92 negative specimens was retested repeatedly on different assay runs.
From January 2003 to April 2005, clinical specimens were originally tested by SDA. GC-positive samples were repeat tested by SDA, and aliquots of the original specimens were then tested by PCR and AC2. For PCR confirmation, 500 μl of the SDA sample was spun at 12,000 × g for 15 min. The supernatant was discarded, and the protocol for PCR swab processing and testing was followed. For AC2 confirmation, 100 μl of the SDA sample was inoculated into an AC2 swab transport tube and tested according to package insert instructions.
From to June 2003 to March 2004, clinical specimens collected from MSM were initially tested by PCR. All positive specimens were repeat tested by PCR. The original specimens (swabs in M4 medium or FCU samples) were then tested by SDA and AC2. For SDA confirmation, 100 μl of the M4 sample was inoculated into the ProbeTec transport tube, or 4 ml of FCU was processed and tested by the SDA protocol. For AC2 confirmation, 100 μl of the M4 sample was inoculated into an AC2 swab transport tube, or FCU was inoculated into an AC2 urine transport tube. A separate 400-μl aliquot of the AC2 inoculated transport tube was tested by AC2 according to the manufacturer's instructions. GC cultures were done on additional swab (rectal) samples.
From June 2003 to May 2007, clinical specimens were originally tested by AC2. All positive specimens were repeat tested by AC2 and AGC. The original specimens (collected in M4) were then tested by SDA. An aliquot (100 μl) of the sample was inoculated into the ProbeTec transport tube, and the protocol for SDA swab processing and testing was followed. For PCR confirmation (FCU only), 500 μl of urine was processed and tested by the Amplicor assay. GC cultures were done on a second swab (pharyngeal and rectal) sample. Patients with a positive glans swab also had an AC2 test done on their FCU sample.
A multicenter clinical evaluation of the AC2 and AGC assays was conducted in North America from January 2003 to May 2003. The results of that clinical trial study were reported previously (2, 8). There were 1,465 women and 1,322 men enrolled in the study. Two randomized urethral swabs and FCU samples were collected from men. Two randomized endocervical swabs and FCU samples were collected from women. Each specimen type was tested by AGC, AC2, and SDA. Thus, three NAATs, each having a different target, were used to test specimens from each subject. The SDA, AGC, and AC2 assays were each considered to be the primary screening tests, and all positive results for each test were then confirmed by both the other tests.
For CDC approaches 1 and 2, approximately 7,987 specimens were screened. The majority of positive samples were collected from men; ~37% were collected from women. The prevalence of GC varied by clinic, ranging from 0.5 to 9.2%. The MSM population had the highest GC prevalence.
Table Table11 shows the results for GC confirmation by repeating the original test on the original specimen (CDC approach 1). All three NAATs had similar results, ranging from 89.3 to 96.4% confirmation. By testing samples positive by SDA a third time and a fourth time, and additional 3% and 4% of the positive results were confirmed, respectively. With AC2, a failure for a sample to retest positive was associated with specimens having low RLU readings (≤650,000). Figure Figure11 shows that 59% of specimens initially positive by AC2 that did not have a positive result upon retest were low-level positive results. Similar results were found with SDA. Again, the majority of repeat failures (Fig. (Fig.2)2) were due to low-level positive results, but failures were also seen at all ranges, even at the high end of >40,000 mota. PCR had two repeat failures. For these repeat failures, inhibition was not detected with the internal controls used for the SDA and PCR assays. When low-level positive results were retested several times by SDA and AC2, we observed variable results (Table (Table2).2). All of the positive results were ultimately confirmed by a different NAAT, but random sequences of positive and negative results were seen with the repeat assays. This pattern was more evident with the SDA assay. With AC2 repeat studies, we were able to use another NAAT for confirmation. By using AGC (Table (Table1),1), 95.7% of samples that tested positive by AC2 were confirmed.
AC2 was also evaluated for reproducibility of negative results. We tested 92 specimens for a total of 552 repeat tests. All samples were negative upon repeat testing, implying close to 100% specificity.
For CDC approach 2, there was a total of 113 originally GC-positive samples by SDA (Table (Table3).3). When different NAATs were used for confirmation, PCR confirmed 78% of positive results, while AC2 confirmed 85% of positive results. Repeat testing was the better strategy, confirming 88.5% of the SDA-positive results. However, by using a combination of repeat SDA, PCR, and AC2, 93.8% of the originally positive results were confirmed. Table Table44 shows samples that were originally positive by PCR were confirmed to a high degree by SDA, AC2, or repeat testing. The use of a combination of both approaches resulted in 100% confirmation of results that were positive by PCR. For AC2, we tested 325 originally positive samples (Table (Table5).5). When different NAATs were used for confirmation, AGC confirmed 95.7% of positive samples, whereas SDA confirmed only 84.6% of positive samples. Repeat testing yielded results similar to those for AGC confirmation. However, by using all combinations of testing, 98.2% of positive results were confirmed.
Tables Tables44 and and55 show confirmations of positive results by PCR and AC2 for different specimen types. A combination of the three CDC approaches confirmed >97% of these positive results. Positive results with FCU and glans specimens were easily confirmed by either repeat testing or retesting with other NAATs except that SDA was less efficient at confirming other positive results by NAAT. The use of culture on an additional swab to confirm a positive result by NAAT with rectal or pharyngeal specimens was not as efficient as the use of another NAAT, as 25 to 60% of samples that were positive by NAAT were not confirmed by culture.
Table Table66 shows results using a second specimen and a different NAAT for confirmation. For all specimen types, AC2 yielded 536 positive results, the AGC assay yielded 560 positive results, and SDA yielded 515 positive results. Both the AGC and the AC2 tests confirmed 95.7% of the SDA positive results. Of the samples that tested positive by AC2, 529 (98.7%) were positive by AGC, but only 493 (92%) were positive by SDA. Of the samples that tested positive by AGC, the same numbers were confirmed, 529 (94.5%) samples by AC2 and 493 (88%) samples by SDA. All NAATs were comparable in confirming positive FCU specimens from men. However, with FCU and swab samples from women, SDA was less efficient at confirming AC2 or AGC results.
The three CDC-suggested approaches for the confirmation of Neisseria gonorrhoeae vary in performance. In general, these results were similar to our results from a previous study that evaluated the same approaches for the confirmation of positive results determined by Chlamydia trachomatis NAATs (13). We found that the most reasonable single approach may be the performance of a repeat test. However, the reliability of this approach is uncertain. Given that PCR and SDA can produce FP results with other Neisseria species, is a NAAT repeat test appropriate for confirmation (6, 19)? An FP specimen could be confirmed and thus interpreted to be a true-positive specimen. This is the major flaw of repeating a test to confirm a GC-positive result. A laboratory cannot determine whether a positive SDA or PCR result upon retest is Neisseria gonorrhoeae or another Neisseria species that has acquired the target.
Our in-house evaluation of Amplicor PCR for the detection of GC in samples from MSM proves this point. We found the pharyngeal swab to be an unsuitable specimen for use with GC PCR because of numerous FP results (16). These FP results were due to cross-reactions to other Neisseria species and resulted in a specificity of only 79% for NAAT. When a repeat test was used to confirm these positive results, 85% (44/52) of these specimens would have been identified as being true-positive results. However, when culture was used with SDA or AC2 to confirm the result, only 12% (6/52), 21% (11/52), and 19% (10/52) of specimens were identified as being true-positive results, respectively. Thus, we do not use this PCR with pharyngeal specimens.
If we assume that a retest does confirm a GC-positive result, a repeat test confirmed 89 to 96% of samples that tested positive by SDA, PCR, and AC2 (Table (Table1).1). As expected, most of the SDA and AC2 repeat failures were due to low-level positive results (Fig. (Fig.11 and and2)2) that generated random positive and negative results upon retesting. This erratic pattern of results was more evident with samples that were positive by SDA than with those that tested positive by AC2 and may be due to low amounts of target in the sample or the presence of inhibitors. SDA is susceptible to transient inhibitors, whereas the AC2 assay apparently is not susceptible because the target capture technology separates target from potential inhibitors. Our results with the internal controls show that the SDA repeat test failures were not due to inhibition. Culler et al. previously reported similar results for the reproducibility of GC SDA (3), and we have also observed the same effect with repeat testing of positive C. trachomatis NAAT results (14). While our previous study confirmed an additional 9% of C. trachomatis-positive specimens with a third repeat SDA, this evaluation confirmed only an additional 3 to 4% of GC-positive results with third and fourth repeat SDA tests.
A 100% specificity of AC2 was found with our repeat testing of initially negative GC samples, suggesting that package insert specificities are not realistic. Higher specificities were also found with retests of negative C. trachomatis NAAT results (14). If manufacturers used discrepant analyses to calculate their performance profiles, then more accurate test specificities would be reported in the NAAT test kits (7, 17). It is apparent that misclassification bias (true-positive result classed as being FP) is greater than the bias introduced by discrepant analysis.
Theoretically, the second CDC approach (to test the original specimen with a different NAAT) is the preferred method for confirmation. Our procedure using the original SDA transport medium was validated for use with AC2 and PCR. However, it is possible that optimal NAAT performance may be obtained only with specimens collected in the transport medium designed for each specific NAAT, and this could occasionally lead to a failure to confirm the test result. The use of an alternate gene target for confirmation was either comparable to or slightly less sensitive than a repeat test. Obviously, the AC2 and AGC assays were better at confirmation than SDA and PCR. The AGC test confirmed 95.7% of our samples that were positive by AC2; Golden et al. also had similar findings (AGC confirmed 94.6% of samples that tested positive by AC2) (9). However, we found that confirmatory testing differed among the specimen types and NAATs. SDA cannot be used to confirm positive AC2 or PCR results from pharyngeal and rectal specimens but can be used to confirm positive AC2 results with glans swabs. We found that all other NAATs were comparable in confirming positive GC PCR results with FCU samples from men. FCU and glans specimens from men may contain large amounts of target. Thus, all the NAATs have good performances. Clearly, the second approach confirmed some positive results that repeat testing did not, and vice versa. When both methods were combined for confirmation, 94 to 100% of all GC-positive NAAT results were confirmed.
The third CDC approach for confirmation is unlikely to be routine, as clinicians are not likely to collect two swabs, nor will they recall a patient, and most laboratories are not performing two different NAAT methodologies. When this type of confirmation is used, the assays used should have equivalent sensitivities. In another study, we found that some NAATs cannot be used to confirm the C. trachomatis results from other NAATs (15). As with CDC approach 2, all GC NAATs were comparable in confirming positive results from urine specimens from men but not from urine specimens or urogenital swabs from women. SDA could not be used to confirm results (swab and FCU specimens from women) from the AC2 or AGC test. However, AGC and AC2 could be used to confirm results from the SDA test.
Falsely GC-positive NAAT results can result from (i) amplicon contamination, (ii) cross-reactions with other Neisseria species, and (iii) laboratory errors (mislabeling and pipetting, etc.) or specimen contamination in the clinic. Unfortunately, confirmatory testing will not correct most of these sources of error. For example, laboratorians should be aware that repeat testing does not work for FP results caused by other Neisseria species, amplicon contamination, or specimen contamination. Indeed, no retest will correct for specimen contamination (as the target has been introduced) or for specimen mislabeling, as the positive result is correct but is incorrectly being attributed to the wrong patient. We found that some NAATs cannot be used to confirm other positive NAAT results but that some specimen types can be confirmed by all NAATs. With these caveats, any guidelines for GC confirmatory testing must be confusing. Fortunately, our results simplify the situation. We have shown that >94% of GC-positive NAAT results were confirmed by two of the CDC approaches. Thus, confirmatory testing of positive GC samples is not necessary for SDA, PCR, and AC2 assays using genital tract specimens.
This work was partially supported by the donation of test kits used for the confirmation of positive specimens from the following manufacturers: Becton Dickinson Co., Gen-Probe Inc., and Roche Molecular Systems.
Published ahead of print on 5 March 2008.