|Home | About | Journals | Submit | Contact Us | Français|
Because self-collected vaginal swabs (VS) are potentially very useful for screening asymptomatic women for Chlamydia trachomatis infection, a multicenter study evaluated that specimen with nucleic acid amplification tests (NAATs). The objective was to determine whether VS are equal to Food and Drug Administration (FDA)-cleared specimens (cervical swabs and first-catch urines [FCU]) for diagnosing genital chlamydial infection. All NAATs then commercially available (October 1996 to October 1999) were used (ligase chain reaction [LCx Probe System; Abbott Laboratories, Abbott Park, Ill.]; PCR [Amplicor; Roche Molecular Systems, Branchburg, N.J.]; and transcription-mediated amplification, [Amplified CT Assay; Gen-Probe Inc., San Diego, Calif.]). NAATs were performed on FCU, urethral, cervical, self- and clinician-collected VS. Sensitivity was compared to isolation using cervical and urethral swabs. Agreement of NAAT results between VS and cervical swabs or FCU was calculated. Specimens from 2,517 15- to 25-year-old asymptomatic women attending clinics at nine different centers were evaluated. Results with self- and clinician-collected VS were equivalent and were at least as good as results with FCU and cervical swabs. Across all sites, summary specificities for all specimens were >99%. Among culture-positive women, NAAT sensitivity with VS (93%) was as high as or higher than NAAT sensitivity with cervical swabs (91%) or FCU (80.6%) or culture of cervical swabs (83.5%). VS are appropriate specimens for diagnosing chlamydial genital tract infection by NAATs. That patients can efficiently collect them offers important benefits for screening programs. It would be beneficial for public health programs if the NAAT manufacturers sought FDA clearance for this specimen.
The introduction of nucleic acid amplification tests (NAAT) for the diagnosis of genital Chlamydia trachomatis infection was a major step forward for laboratory-based studies on this important pathogen. Not only are the NAATs more sensitive than any previous existing diagnostic test, but they are also extremely specific (2, 6, 12, 13, 16). This means that they can be used for screening low-prevalence populations and provide results with high predictive values. The first studies of commercially available NAATs showed that they could be applied to first-catch urines (FCUs) from symptomatic and asymptomatic men (1, 6). A major breakthrough was the observation that NAATs could be used with noninvasive specimens, such as FCU from women. The sensitivity was similar to that obtained with cervical swabs (8, 11). This sensitivity results both from detection of the small number of chlamydiae in the urethra and from organisms present in vaginal secretions that enter the urine specimen during collection. That noninvasively collected specimens can be used for the diagnosis of chlamydial infections in both men and women makes possible true population-based prevalence surveys and broad-based screening approaches to control chlamydial infections. Screening, or the ability to diagnose asymptomatic infections, is important for control of bacterial sexually transmitted diseases in general and particularly for C. trachomatis, which often causes asymptomatic infections.
Unfortunately, further experience found some problems in testing FCUs with NAATs. Shortly after the first publications showing successful results, a negative report appeared stating that ligase chain reaction (LCR) could not be used with FCUs from pregnant women because of poor sensitivity (I. P. Jensen, P. Thorsen, and B. R. Moller, Letter, Lancet 349:329-330, 1997). This in fact was not the case, since subsequent studies found LCR to be highly effective with FCUs from pregnant women (3, 9). In retrospect, the negative report was probably due to unrecognized problems in processing urine specimens. The processing of urine for some NAATs involves a centrifugation step to separate the chlamydial elementary bodies from the urine. If too much urine is allowed to remain on the sediment, it can result in inhibition of LCR, because the phosphate in urine is a powerful inhibitor of the LCR reaction (10). Subsequent studies have found that vigorous aspiration of the supernatant after centrifugation could disturb the sediment, resulting in its loss, to provide another source of false-negative NAAT results (D. H. Martin and C. Cammarata, Abstr. 13th Meet. Int. Soc. Sex. Transm. Dis. Res., abstr. 385, 1999). In a multicenter trial this happened with both LCR and PCR (7).
Soon after FCUs from women were validated for screening purposes, a number of investigators reported that vaginal or vulval swabs were also useful diagnostic specimens (5, 15, 17). The vaginal swab specimens offered important advantages for transport and processing compared to FCU. In some comparison studies there were slightly better sensitivities with vaginal swab specimens than with FCUs, perhaps owing to greater stability. In light of the processing errors that have been identified with urines, it cannot be ascertained whether these are true differences in sensitivity or another reflection of problems associated with the handling of urine specimens in the laboratory.
Despite these advantages of the vaginal swab as a specimen, no commercial NAAT test kit has been cleared by the Food and Drug Administration (FDA) for use with vaginal swabs. Indeed, vaginal swabs have never been indicated for use with any commercial diagnostic test for C. trachomatis. Thus, for manufacturers to get these tests approved for use with vaginal swabs, they would be breaking ground by indicating a new specimen type. This might require a premarket application, which is an expensive undertaking. An alternative market pathway might be to seek a less expensive premarket notification [510(K)] that requires establishing substantial equivalence to a previously cleared indicated specimen type.
Because many investigators in the field felt that vaginal swab specimens would be useful in screening for chlamydial infections, a multicenter study was designed to evaluate vaginal swabs as specimens. This study was planned when only three NAATs had been cleared by the FDA: PCR, LCR, and transcription-mediated amplification (TMA). A proposal was made to the manufacturers of these three tests seeking support. We hoped such a study could determine whether vaginal swabs would be truly useful specimens, and we hoped to generate data that might be useful for FDA clearance to consider an evaluation of the vaginal swab indication based upon equivalence. Each manufacturer was asked to contribute $150,000.00, and a small grant was obtained from the Centers for Disease Control and Prevention to coordinate the study and analyze the data. Nine laboratories, each experienced in evaluating diagnostic tests, were selected to participate. Each NAAT was performed by three of the nine sites. There was no effort to compare the results of the different NAATs; the sole emphasis of these studies was to compare the vaginal swabs to indicated specimen types with FDA-cleared test kits.
The laboratories performing PCR tests were at Boston University, Louisiana State University in New Orleans (LSU), and Indiana University in Indianapolis. The sites performing LCR tests were at University of Washington at Seattle (UW), University of Alabama at Birmingham (UAB), and Johns Hopkins University (JHU), in Baltimore. The sites performing TMA were at McMaster University at Hamilton, Ontario, Canada (Hamilton), State University of New York at Brooklyn (SUNY), and University of California, San Francisco (UCSF). Protocols were approved by each institution's local review board.
Patients participating in the study were nonpregnant, 16- to 25-year-old women who were appearing at a sexually transmitted disease, obstetrics/gynecology, or family planning clinic for routine examinations or birth control advice between October 1996 and October 1999. Exclusions from the study were the following: treatment with antibiotics within the last 30 days; attending clinic because of symptoms; or having a male partner who was being treated because of genital symptoms. These highly restrictive criteria were selected to focus on evaluation of specimens from the population that would most likely be a target for screening or population-based study of chlamydial infections.
After informed consent was obtained, the women were given a diagram showing the method for inserting the vaginal swabs (copies are available from the senior author, Julius Schachter) and verbal instructions as to how to collect the swab. Patients were asked to insert the swab into the vagina approximately 4 to 5 cm and then to rotate it several times before placing it into a capped tube. They were also given a plastic container and asked to collect 25 to 30 ml of first-void urine. The first specimen collected was the self-obtained vaginal swab for NAAT, followed by the FCU specimen for NAAT. Then, a clinician obtained a vaginal swab for NAAT by following the same instructions given to the patients, before performing a pelvic examination. Two distal urethral swabs for chlamydia culture and NAAT were then collected, followed by insertion of a speculum. Vaginal specimens were then collected for diagnosis of vulvovaginitis, if indicated. A cervical swab was then collected for gonococcal culture, and then two cervical swabs, one for culture and one for the NAAT, were collected for C. trachomatis testing. The last specimen collected was for Pap smear, if indicated at the time of this examination. The order of collection of the urethral and cervical swabs was alternated for culture or NAAT on a patient basis.
All of the participating laboratories have had considerable experience doing chlamydial cultures (TC), and each laboratory performed TC using its standard procedure. Isolation attempts were performed on swabs (processed separately) obtained from the cervix and urethra. The three NAAT technologies commercially available at the time of the study were used (LCR [LCx Probe System; Abbott Laboratories, Abbott Park, Ill.]; PCR [Amplicor; Roche Molecular Systems, Branchburg, N.J.]; and transcription-mediated amplification [Amplified CT Assay; Gen-Probe Inc., San Diego, Calif.]). NAATs were performed on an FCU specimen, cervical and urethral swabs, and on both patient-collected and clinician-collected vaginal swabs following manufacturers' instructions.
Given the multiplicity of specimens, there are many ways to calculate sensitivity and specificity (7). However, because the major goal of this study was to compare vaginal swab specimens with other specimens tested by the same technology, we focused the analyses on the agreement between positive results with the vaginal swab and the FDA-cleared specimens (cervical swab or FCU). Sensitivity was also calculated on the basis of specimens collected from patients who were culture positive at any site.
Specificity was calculated based on two assumptions: that results with specimens that were uniquely positive are false positives, and that multiple positive results obtained from the same patient are true positives.
A total of 2,517 women were tested at the nine sites. There were marked differences in the number of women tested at each site, with a range of 56 to 789 (Table (Table1).1). Each of the NAATs was tested with specimens from at least 500 women, and there were at least 48 culture-positive women for any NAAT comparison.
The overall chlamydial prevalence by isolation was 9.6% (242 of 2,517). The testing by TC of a second specimen (urethra), in addition to the usual cervical culture, increased the number of culture positive women by 19.8% (40 of 202). Of the 242 culture-positive specimens, 40 (16.5%) were positive only at the urethra, 111 (45.9%) were positive at both urethra and cervix, and 91 (37.6%) were positive only at the cervix. Thus, 83.5% of all culture-positive patients were positive at the cervix compared to 62.4% at the urethra (Table (Table22).
Across all study sites the highest number and percentage of positive results was seen with the vaginal swab specimens. The numbers of positive results obtained with self-collected and clinician-collected vaginal swabs were very similar (Table (Table11).
Using a positive culture as the “gold standard,” the cervical, urethral, and vaginal swab sensitivities by NAAT were all higher (90.9 to 93%) than that of cervical culture (83.5%). The sensitivity of the cervical swab was >89% with each NAAT, with some variation (as expected) among the testing locations (Table (Table2).2). Similar sensitivities were seen at 93.4% (226 of 242) for self-collected vaginal swabs and 93.0% (225 to 242) for the clinician-collected vaginal swabs. Sensitivity of FCU was somewhat lower, with more site-to-site variation being seen. The overall FCU sensitivity was heavily impacted by a relatively poor result seen at SUNY, where only 58 of 89 culture-positive specimens (65.2%) were positive by NAAT on FCU. This laboratory was using TMA, and a technical error was subsequently detected in the FCU processing being performed there, which artificially reduced the sensitivity. This will be discussed in greater detail below.
There was good agreement between the positive results by NAAT on self-collected vaginal swabs and positive results obtained with the same NAAT by cervical swabs, 89.0% (252 of 283) (Table (Table3).3). Similarly, if a women's FCU tested positive, her self-collected vaginal swab was likely to be positive, 87.1% (222 of 255). Although the results were not the same at all sites, there was a tendency for there to be less agreement between self-collected vaginal swabs and either FCU or cervical swabs tested by PCR than by either LCR or TMA (Table (Table33).
The greatest variation in positive results seen among these specimens was with FCU. At three sites (SUNY, LSU, and Indiana University) there were markedly fewer FCU positives than vaginal swab positives. At one of these sites (LSU) an error in processing the FCU specimens was detected during the conduct of these studies. Vigorous aspiration of the supernatant after centrifugation of the urine resulted in disruption of the pellet and discarding of the sediment containing the desired target. At the other two sites technicians missed their training in urine processing. Where aliquots of FCU were frozen, retests of the false-negative specimens yielded positive results in all instances.
Specificities were estimated by assuming that women who had no positive tests, or only one positive test, were truly negative. The specificity of the vaginal swab specimen was similar to specificity with the other specimens (Table (Table4).4). In general the specificities were greater than 99%. Of the 2,201 women who had positive results with one or no specimens, there were three each with uniquely positive cervical or urethral cultures (giving the culture systems a calculated 99.9% specificity). The specificities of the NAATs were also high with any specimen (99.1 to 99.4%) (Table (Table4).4). There was relatively little laboratory-to-laboratory variation in terms of overall specificity. The lowest specimen specificity was 98.1% with FCU in LCR, but this was due to an excess of false-positive results seen with LCR at UW. The specificity results were very similar with self-collected and clinician-collected vaginal swabs. There were between 13 and 20 uniquely positive NAATs for the different anatomic sites (13 with self-collected vaginal swabs and 20 with FCU and urethral swabs).
There were not many discrepant results, since overall there were 83 uniquely positive specimens among the >12,000 NAATs that were performed, meaning an implied minimum specificity of 99.2% for the entire study. Attempts to evaluate these unique positives were made, when specimens were available, by direct fluorescent antibody or alternate amplification tests. The use of another approved amplification test would obviously be the best way of evaluating these tests, but that was not always possible. Of the 83 discrepant results, 40 specimens were retested by a different cleared NAAT or by direct fluorescent antibody procedures, and 30 were found to be positive by the presence of chlamydial antigens (7 specimens) or genes (23 specimens). (These data are not included in any calculation but simply make the point that when any single amplification test is evaluated there will be unique positives that cannot be confirmed by culture, or non-NAAT formats, that will be confirmed as truly positive by other NAATs).
The overall prevalence of positive test results with each of the specimens in each of the NAATs is presented in Table Table5.5. The lowest prevalence of positive results was obtained with the TC systems. Any of the NAATs, applied to any of the specimens, identified more patients as being infected than did TC. The vaginal swab specimens had the highest yield of positive results.
The initial goal was to have 500 asymptomatic young women tested at each of the institutions. At some clinics recruitment was difficult because asymptomatic women attending for routine exams were a small minority of the clientele. Thus, the goal of having 1,500 women tested by each NAAT (LCR, or PCR, or TMA) was not met.
As expected from published studies, the results with vaginal swabs were essentially equivalent to results obtained with cervical swabs by NAAT (5, 15, 17). For purposes of this study, sensitivity was calculated based on culture-positive patients (cervical or urethral). All NAATs were more sensitive than cervical culture with swab-collected specimens. It is noteworthy that there was no difference between results obtained with clinician-collected and patient-collected vaginal swabs. Clearly patients are equally efficient in collecting this specimen, when provided with adequate instruction. The equivalence in results should provide confidence regarding the use of self-collected specimens in population-based surveys where specimens are collected away from medical facilities and then submitted for testing.
The ultimate goal of diagnostic testing is to identify infected individuals who require treatment. The overall prevalence of positive results by the different testing combinations is presented in Table Table5.5. There was a considerable increase in the number of infected individuals identified by NAATs compared to culture. In fact, the use of vaginal swabs would result in treatment of approximately 50% more women than would be treated based on use of cervical cultures. This obviously is expected from previous evaluations, which have found NAATs to be more sensitive than TC. The overall prevalence of positives by the dual-culture system was 9.6% for culture (only 8.1% by culture from the cervix, which typically is the sampled site). By NAAT on cervical swabs, approximately 11.4% of the specimens were positive, for a 40% increase in women who would be identified and treated. The extremely high specificity of the NAATs has been documented in previous studies. The increment in the individuals who test positive primarily reflects identification of truly infected individuals who are falsely negative in the older technologies, such as TC, antigen detection, or nonamplified nucleic acid hybridization tests.
What is important for the purposes of this study is that the results with the self-collected vaginal swabs, or the clinician-collected vaginal swabs, were in excellent agreement with positive results seen with cervical swabs (Table (Table3).3). There were a large number of vaginal swab specimens that tested positive despite negative cervical cultures. These results were typically in agreement with other NAATs from the same patient, and thus, there was no excess of uniquely positive (i.e., false-positive) vaginal swabs. From these data it is evident that the vaginal swab is at least as good as any other specimen for detecting chlamydial infection by NAATs. (Parenthetically, it should be noted that a similar argument could be made for the use of vaginal swabs for diagnosis of gonococcal infection where good results have also been obtained in other studies) (4). Besides excellent sensitivity and specificity, the vaginal swab has a number of advantages over FCU. FCU is more difficult to transport. There are problems with leakage, bulk, and specimen disposal. In addition, testing FCU is more expensive due to the extra processing (centrifugation, resuspension, etc.), and results are more variable due to potential processing errors.
In this study, some laboratories had lower positivity with the FCU than with the cervical specimens. Some were identified as being due to the processing errors described above. In two of these laboratories, the errors were apparently because the technicians doing the tests had not had appropriate training in the specific test technology. Technicians in both laboratories had missed parts of the on-site training that is a prelude to any of these clinical trials. When available, stored specimens that were falsely negative were retested, and they were found to be positive. These observations make the point that laboratories cannot take an “off-the-shelf” approach to NAATs. It is imperative that technicians receive adequate training in using these tests. One laboratory had an excess number of false-positive results with urine specimens in the LCR tests. The reason for this is unknown. These problems were detected here only because of the design of the trial and from the multiple numbers of specimens being processed from each patient. In routine clinical practice there is no way to determine whether the urine processing errors that we encountered in our study are being duplicated. However, we suspect that these errors do occur and cause false-negative results.
Obviously, FCU will not be dispensed with, since it will still be the specimen of choice for screening asymptomatic males and indeed may be the specimen of choice for diagnosis with symptomatic males. However, urine processing errors do require attention. The problem should be publicized, information about this should be more widely circulated by the manufacturers and public health laboratories, and there should be better quality assurance and training programs to minimize the problems.
One of the most beneficial aspects of NAATs in diagnosing chlamydial and gonococcal infections is the fact that they can be used on noninvasive collected specimens. Thus, they are suited for broad-based prevalence surveys, population screening, and long-term follow-up studies of individuals who may be inconvenienced by coming in for routine examinations. The cost/benefit analysis of doing testing without performing pelvic examinations is markedly in favor of a simple self-collected specimen testing approach (14). A pelvic exam adds considerably to the cost. (Where there are no medical indications for pelvic examinations—this is clear. Whether it should be done on a routine basis is open to discussion, but we strongly endorse pelvic examinations for women with genital tract symptoms). To address the issue of whether important information is being lost by not doing routine physicals on these women, clinical findings and other laboratory results on all of the women enrolled in this study were systematically collected. The results will be the subject of a future manuscript.
It is clear that the vaginal swab specimen has much to offer over an FCU. It is simple to collect and easy to transport and process at the laboratory (and thus less expensive). The results obtained with vaginal swabs collected by the patients were equal to the results obtained with specimens collected by clinicians and comparable to the results obtained with the cervical swab, which is currently considered to be the specimen of choice.
We end this paper with a plea. As researchers and public health advocates, we want vaginal swabs to be available to us as a routine diagnostic specimen. The vaginal swab is as good a specimen as others that are already approved. Manufacturers may be reluctant to support the more-expensive studies to generate data to be submitted to the FDA, feeling that breaking ground for a new specimen would require an approval process that would be expensive. To get approval for a test where there is a previously approved similar test available requires a simple demonstration of equivalence. Hopefully, manufacturers and the FDA could build on the data presented here and in other published studies to develop cost-effective protocols that would lead to rapid approval of the use of vaginal swabs for diagnosis of chlamydial infections. The future health of the populations at greatest risk from these infections will be benefited greatly.
This study was supported in part by each of the manufacturers of the diagnostic tests (Amplicor, Roche Molecular Systems, Branchburg, N.J.; LCx Probe System, Abbott Laboratories, Abbott Park, Ill.; and Gen-Probe Amplified CT Assay, Gen-Probe, Inc, San Diego, Calif.). In addition, the Centers for Disease Control and Prevention provided funds for data analysis.