In the United States, human papillomavirus (HPV) DNA testing is now used (12
) (i) as a triage test for determining whether a woman with atypical squamous cells of undetermined significance (ASC-US) needs immediate colposcopic evaluation and (ii) as an adjunct to cervical cytology, “cotesting,” for cervical cancer screening of women 30 and older. HPV assays must balance clinical sensitivity with specificity to minimize both false-negative and false-positive results (10
). Validation studies are needed to demonstrate this balance. In addition, the detection of some individual HPV genotypes, specifically HPV16 and HPV18, the two most carcinogenic HPV genotypes, may be clinically useful for those being screened with HPV in deciding which of the many HPV-positive women with negative cytology need immediate colposcopy (8
). With these goals in mind, we evaluated a new HPV test developed by BD Diagnostics (Sparks, MD) for its analytic and clinical performance using a random sample of archived enrollment specimens from the A
SCUS and L
ow-Grade Squamous Intraepithelial Lesion (LSIL) T
tudy (ALTS) group.
Study design and population.
ALTS (1997 to 2001) was a multisite, randomized clinical trial comparing three management strategies (immediate colposcopy [IC], HPV triage, or conservative management [CM]) for women referred for ASCUS (n
= 3,488) or LSIL (n
= 1,572) conventional cytology (1
). (ASCUS under the 1991 Bethesda system [9
] was slightly more inclusive, particularly of probable reactive changes and atypical squamous cells for which high-grade intraepithelial lesion cannot be ruled out [ASC-H], than the ASC-US category of the 2001 Bethesda system [15
].) The National Cancer Institute and local institutional review boards approved the study, and all participants provided written, informed consent.
At enrollment and follow-up visits over the 2-year duration, all women underwent a pelvic examination with collection of two cervical specimens, the first specimen in PreservCyt for ThinPrep cytology (Cytyc Corporation, Marlborough, MA; now Hologic) and the second in specimen transport medium (STM; Digene Corporation, Gaithersburg, MD; now Qiagen). Women in all three arms of the study were reevaluated by cytology every 6 months during the 2 years and sent to colposcopy if the cytology result was high-grade squamous intraepithelial lesion (HSIL). An exit examination with colposcopy was scheduled for all women. We refer readers to other references for details on randomization, examination procedures, patient management, and laboratory and pathology methods (1
Hybrid Capture 2 (HC2; Qiagen) testing was performed on the PreservCyt specimen throughout the trial in accordance with the manufacturer's instructions (13
). Line Blot Assay (LBA; Roche Molecular Systems, Alameda, CA) for the detection of 27 or 38 HPV genotypes and Linear Array (LA; Roche) for detection of 37 of 38 HPV genotypes detected by LBA (excluding HPV57) were conducted on STM specimens as previously described (4
BD Viper HPV assay testing was performed on aliquots of 500 randomly selected enrollment STM specimens which had been archived at −80°C. We restricted the selection to using only enrollment STM specimens with 3 or more aliquots remaining to preserve ALTS as a resource for future validation studies. Testing was done masked to the previous test results and clinical outcomes.
An aliquot of 25 μl of each specimen was diluted with 475 μl of deionized water and then combined with 1.5 ml of a proprietary BD HPV diluent. The sample input volume was based on pilot cell-spiking experiments. Equal numbers of cells were spiked into 1 ml of STM and 20 ml of ThinPrep medium, and the responses observed with 5, 10, 20, and 40 μl of STM extracted using our standard assay standard operating procedure (diluted to 0.5 ml/sample) were compared to the responses observed with 0.5 ml of ThinPrep medium for all channels of the assay. The 25-μl sample input volume chosen was designed to give a cell loading approximately equivalent to that observed for 0.5 ml of ThinPrep medium.
The resultant mixture was heated at 120°C for 25 min and allowed to cool to room temperature. DNA was then extracted using BD Viper XTR ferric oxide (FOX) particle DNA binding and magnetic extraction (6
). The DNA was eluted in 400-μl and 50-μl aliquots of the DNA solution that was used to rehydrate each of three PCR microwells (AbGene; Thermo Fisher Scientific, Inc.) containing a dried-down master mix and HotGoldStar Taq
polymerase (Eurogentec S.A., Belgium). Each of three wells contains four fluorescent real-time PCR probes for 4 separate reactions/channels (). In toto
, the 12 reactions enabled the detection of all 13 carcinogenic genotypes of HPV and HPV66, with an internal beta-globin control in each well. Rehydrated PCR microwell strips were sealed with an adhesive plate seal (Applied Biosystems) before being placed in a Stratagene Mx3005P (Agilent Technologies) thermocycler (95°C enzyme activation step for 15 min followed by 43 cycles of 95°C for 30 s and 55°C for 60 s). DNA amplification and detection were performed simultaneously using real-time PCR detection, and the signal output from each of the four channels (four colors) per well was read independently in each of the three assay wells. HPV-positive specimens were identified using a cycle threshold algorithm method.
The positive cut point was established in a separate study (unpublished results) in which the results for the BD assay on paired cytology specimens in both ThinPrep and SurePath (BD) media were compared to the results for the HC2 assay on the same specimens. ROC analysis was then used to determine the cutoff which best matched the HC2 results, with LA being used for discordant testing and for adjudication of HC2-positive specimens whose positivity was caused by low-risk HPV genotypes due to cross-reactivity with those types as previously reported (5
To ascertain that there were no meaningful differences between selected and unselected specimens, we compare the distributions between groups for study design parameters (referral Pap interpretation, clinical center, and study arm) by Fisher's exact test and enrollment age by the Kruskal-Wallis test. We compared the BD HPV assay to HC2 for detection of carcinogenic HPV DNA (including HPV66, which is targeted by BD and detected by HC2 via cross-reactivity [5
]) and to LBA and LA for detection of (i) 7 individual carcinogenic HPV genotypes (HPV16, -18, -31, -45, -51, -52, and -59), (ii) a pool of HPV33, -56, -58, and -66, (iii) a pool of HPV35, -39, and -68, and (iv) any carcinogenic HPV genotype plus HPV66 by use of agreement statistics (kappa value, percent agreement, and percent positive agreement). An exact version of the McNemar chi-square test was used to test for statistically significant differences (P
< 0.05) in testing positive. We also examined the HPV genotype results by LBA and LA individually and categorized the results hierarchically (carcinogenic > noncarcinogenic > negative) for discordant HC2 and BD results (HC2+
). We also calculated the values for clinical performance parameters (sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) for detection of cervical intraepithelial neoplasia grade 3 (CIN3) for the BD HPV assay and compared them to the values obtained with HC2.
There were no differences between women whose enrollment specimens were selected for testing by the BD assay and women whose specimens were not selected in (i) percentage of women referred to ALTS for an ASCUS (versus LSIL) Pap smear (71% versus 69%; P = 0.3), (ii) median age at enrollment (25 years versus 25 years; P = 0.8), (iii) distribution across the clinical center (P = 0.7), or (iv) the management arm to which the patient was randomized for the trial (P = 0.7).
shows the paired BD and HC2 results for 473 women for detection of carcinogenic HPV DNA, overall and stratified by worst 2-year histologic diagnosis; 2 (0.2%) were without BD results, and 25 (5%) were missing HC2 results (241 of 5,060 [4.76%] women enrolled into ALTS were missing enrollment results) from the enrolment testing. Overall, there was very good agreement between the two tests, with an 87.7% total agreement, an 81.4% positive agreement, and a kappa of 0.75. There were no statistically significant differences in overall percent positivity between the two tests (P = 0.4). Similar patterns of agreement were observed for the stratum defined as worst 2-year histological outcome. The clinical performance of the BD assay (sensitivity = 90.0%; specificity = 43.9%; PPV = 12.9%; and NPV = 97.9%) for detection of a 2-year worst histologic outcome of CIN3 was comparable to the performance of HC2 (sensitivity = 87.5%; specificity = 41.8%; PPV = 12.2%; and NPV = 97.3%). Specimens that tested HC2−/BD+ were more likely to be positive for carcinogenic HPV genotypes, whereas HC2+/BD− specimens were more likely to be positive for noncarcinogenic HPV genotypes (excluding HPV66) as measured by LBA (P = 0.01) or by LA (P = 0.009) (see the table in the supplemental material).
shows the comparison of test positivity for BD to that for LBA and LA for a subset of 7 HPV genotypes (HPV16, -18, -31, -45, -51, -52, and -59) detected individually by all three assays. It also shows the comparison for detection of a pool of HPV33, -56, -58, and -66 and of HPV35, -39, and -68 by BD versus detection of any of those types in each pool by LBA or by LA and for detection of any carcinogenic HPV genotype. The analysis was restricted to the 371 (74.2%) of the 500 selected women for whom we had test results available for three assays. In general, all three tests had similar results for test positivity for individual HPV genotypes, with BD generally having intermediate test positivity relative to LBA (lowest) and LA (highest). There were two notable exceptions: BD was significantly more likely to test positive for HPV51 than LBA (P = 0.0001) and LA (P = 0.01) and was less likely to test positive for HPV59 than LA (P = 0.04). For any carcinogenic HPV genotype, results for LA were 57.7% positive, those for BD were 55.0% positive, and those for LBA were 50.4% (P = 0.009 for BD versus LBA).
Here, we demonstrated that the clinical performance of the BD assay was clinically comparable to HC2, a well-validated clinical test and benchmark for clinical carcinogenic HPV DNA detection (10
). The analytic sensitivity of the BD assay for the two most important HPV genotypes, HPV16 and HPV18, appeared comparable to that of LBA and LA, suggesting that its performance for detecting these HPV genotypes in a clinical setting, i.e., deciding which carcinogenic HPV-positive, Pap-negative women need immediate colposcopy (8
), may be good.
We note a few limitations in our analysis. First, we have assumed that discrepant results between any two assays are due to a false-negative result for one assay rather than a false-positive result for another. No two assays will be perfectly concordant. Using yet another assay, such PCR-amplified DNA sequencing, may have confirmed some of the discordant results as true positive/false negative (versus false positive/true negative) but would be unlikely to confirm all of the results, since no test is perfectly sensitive either. And sequencing does not handle multiple HPV infections, which is common in this population. While there is no gold standard HPV genotyping assay, LA and LBA are commonly used, well-validated assays. Importantly, there was very good concordance with HC2 for a clinically relevant endpoint of CIN2 or CIN3, which is the critical criterion for a clinical assay.
Second, we acknowledge the use of frozen (−80°C), archived specimens that have been stored for over 10 years. Thus, it is possible that the specimen DNA underwent some small degree of degradation, affecting the results. If so, it seems unlikely that this would be specific for HPV DNA versus host DNA. Use of an internal control helps to control for this possibility, and only two specimens were rejected because of any negative internal control.
We conclude that the BD assay is a promising new clinical test for detection of carcinogenic HPV DNA and warrants further evaluation. Additional clinical studies are currently being conducted with histology-confirmed endpoints to further refine the clinical cutoff for the assay prior to the initiation of pivotal clinical trials for regulatory body submission.