|Home | About | Journals | Submit | Contact Us | Français|
We compared the clinical performance of the PapilloCheck human papillomavirus (HPV) assay with that of the GP5+/6+-PCR method with an enzyme immunoassay readout (GP5+/6+-PCR-EIA) for the detection of high-risk HPV (hrHPV) types by the use of cervical samples originating from women in a population-based by the use of cervical screening cohort tested by combined cytology and GP5+/6+-PCR-EIA (POBASCAM trial). Specimens from a random sample of 1,437 controls (women ages 40 to 60 years with normal cytological findings and without evidence of cervical intraepithelial neoplasia grade 2 or higher [≥CIN2] within up to 8 years of follow-up) and 192 cases (women ages 30 to 60 years in whom ≥CIN3 was detected within up to 3 years of follow-up) were subjected to analysis by the PapilloCheck method. When all 17 (probably) hrHPV types were taken into account, the PapilloCheck assay had a clinical sensitivity for the detection of ≥CIN3 of 96.4% (185/192 samples; 95% confidence interval [CI], 93.7 to 99.7) and a clinical specificity for the detection of ≥CIN2 of 96.3% (95% CI, 95.3 to 97.3). After restriction of the analysis by the PapilloCheck assay to the 14 hrHPV types targeted by GP5+/6+-PCR-EIA, the clinical sensitivity and clinical specificity values were 95.8% (95% CI, 92.8 to 98.8) and 96.7% (95% CI, 95.7 to 97.7), respectively. By comparison, these values were 96.4% (95% CI, 93.9 to 98.9) and 97.7% (95% CI, 96.9 to 98.5), respectively, for the GP5+/6+-PCR-EIA. When all 17 (probably) hrHPV types were included in the analysis, noninferiority score testing revealed that the clinical sensitivity of the PapilloCheck assay for the detection of ≥CIN3 was noninferior to that of the GP5+/6+-PCR-EIA (P < 0.0001), but the clinical specificity of the PapilloCheck assay for the detection of ≥CIN2 was inferior to that of the GP5+/6+-PCR-EIA (P = 0.08) when lower bounds of 90% for sensitivity and 98% for specificity were used. When the analysis was restricted to the 14 hrHPV types targeted by the GP5+/6+-PCR-EIA, both the clinical sensitivity and the clinical specificity of the PapilloCheck assay were noninferior to those of the GP5+/6+-PCR-EIA (noninferiority score test; P < 0.0001 and P = 0.007, respectively). Thus, when the findings obtained for the 14 hrHPV types detectable by the GP5+/6+-PCR-EIA are considered, the PapilloCheck assay is clinically compatible with the GP5+/6+-PCR-EIA.
It has been universally accepted that cervical carcinoma and its closest preinvasive precursor lesion (high-grade cervical intraepithelial neoplasia [CIN] lesions) are caused by a persistent infection with one or more of about 14 high-risk human papillomavirus (hrHPV) types (10, 15). Cervical cancer screening programs rely on the detection and treatment of cervical precancerous (high-grade CIN) lesions and treatable cancer in order to reduce the mortality from this disease. The results of several large population-based randomized screening trials indicate that compared to cytology, testing for about 14 hrHPV types results in marked reductions in the incidence of high-grade CIN lesions and cervical cancers among women who test negative at follow-up (1, 4, 13, 16-18). This argues for the implementation of testing for hrHPV in cervical screening programs at increased intervals.
The tests for hrHPV that were successfully used in these trials are the commercially available, FDA-approved hrHPV Hybrid Capture 2 assay (hc2) and the consensus primer GP5+/6+-PCR with an enzyme immunoassay (EIA) readout (GP5+/6+-PCR-EIA) (1, 4, 13, 16-18). Both assays target hrHPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68, whereas HPV type 66 (HPV-66) is additionally targeted by the GP5+/6+-PCR-EIA but is detected by hc2 as a result of cross-hybridization. Given their better performance characteristics relative to those of cytology, these tests can be considered clinically validated for use for cervical screening (14).
It has become clear that different assays for hrHPV display different clinical sensitivities and clinical specificities for high-grade CIN lesions and cervical carcinoma (8, 9, 14, 20), and consequently, hrHPV tests different from hc2 and GP5+/6+-PCR-EIA are not necessarily useful for cervical screening purposes. In order to facilitate the acceptance and suitability of novel hrHPV assays for cervical screening purposes, guidelines describing the requirements for the use of HPV tests for primary screening have recently been provided by a North American-European collaboration (14). These guidelines propose the use of a clinical validation strategy, based on analysis of the equivalence of the result of an assay relative to that of a clinically validated reference HPV test (i.e., hc2), with samples that originate from a population-based screening cohort, as assessed by the use of a noninferiority score test (14). By this approach, candidate tests for the detection of HPV DNA can be validated for application in cervical screening programs without the need to perform large, prospective screening trials.
A relatively novel test for the detection of HPV is the Greiner Bio-One PapilloCheck assay (7, 11). This assay is a broad-spectrum PCR-based method that uses a consensus primer set that targets the E1 region of HPV DNA and that allows the simultaneous detection and genotyping of 25 different HPV types (i.e., 15 hrHPV types (HPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 68, 73, and 82), 2 probably high-risk HPV genotypes (HPV types 53 and 66), and 8 low-risk HPV (lrHPV) genotypes (HPV types 6, 11, 40, 42, 43, 44/55, and 70) by DNA chip technology (7, 11). In the study described here, we assessed the clinical performance of the PapilloCheck assay relative to that of the GP5+/6+-PCR-EIA for the detection of hrHPV in cervical scraping samples collected during the course of the POBASCAM randomized-controlled cervical screening trial (3-8). In addition, a comparative genotyping analysis was performed after genotyping of the GP5+/6+-PCR products of GP5+/6+-PCR-EIA-positive samples by a reverse line blot (RLB) assay.
For this study we used cervical scrapings collected during the baseline round from women in the intervention group of the population-based randomized-controlled implementation trial POBASCAM. The women in this trial were screened by combined cytology and hrHPV GP5+/6+-PCR-EIA testing for 14 hrHPV types (5, 6). A detailed description of this trial, including the referral policy and follow-up procedure, has been published previously (2-4, 6). For this study, samples comprised a representative set of 192 cervical specimens from women (median age, 34 years; age range, 30 to 60 years) who had histologically confirmed CIN3 lesions or worse (≥CIN3) detected within a median follow-up time of 5.3 months (range, 0 to 34.6 months) from the time of sample collection and detected by either the GP5+/6+PCR-EIA or cytology, or both (these are referred to as cases). A random sample of 1,473, cytomorphologically normal cervical scrapings were also obtained from women over 40 years of age (median age, 49 years; age range, 40 to 60 years) without a high-grade CIN lesion or worse (CIN lesion grade 2 or higher [≥CIN2]) within a median follow-up period of 58.7 months (range, 37.2 to 99.5 months) from the time of sample collection (these are referred to as the controls). Informed consent was obtained from all study participants, and this study followed the local ethical guidelines of the medical center.
In the POBASCAM trial, cervical samples were subjected to GP5+/6+-PCR followed by EIA readout (4, 6). At a later stage, the GP5+/6+-PCR products of the GP5+/6+-PCR-EIA-positive samples were subjected to genotyping by RLB analysis (22). Briefly, after a conventional cervical smear was made on a glass slide, the remaining material of the cervical scraping was collected for testing for hrHPV by placing the brush in 5 ml sterile phosphate-buffered saline (PBS) and 0.05% Merthiolate. Upon arrival in the laboratory, the samples were spun down and the pellets were resuspended in 1 ml 10 mM Tris-HCl (pH 8) and stored at −80°C for subsequent analysis for HPV. Aliquots (100 μl) were boiled for 10 min and cooled on ice, and 10 μl of the extracts was tested by GP5+/6+-PCR followed by EIA readout for 14 hrHPV types (i.e., HPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68), as described before (6). The EIA-positive GP5+/6+-PCR products were genotyped by the RLB assay, according to previously described protocols (22). For the purpose of this study, the GP5+/6+-PCR-EIA and RLB assay procedures were repeated with DNA extracted from samples that originally displayed discordant results between the PapilloCheck assay and the GP5+/6+-PCR-EIA.
Of the selected cervical scrapings, 1/10th of the original sample was used for DNA extraction with magnetic beads (Macherey-Nagel, Germany) and an automated DNA extraction platform (Microlab Star; Hamilton Robotics, Switzerland), according to the recommendations of the manufacturer. The isolated DNA was subjected to the PapilloCheck assay (Greiner Bio-One, Germany), according to the standard protocols provided with the kit. Briefly, the PCR amplifies a 350-bp fragment within the E1 region of the HPV genome by using a broad-spectrum consensus primer set and simultaneously amplifies a region within the human ADAT1 gene (adenosine deaminase, tRNA specific 1) as an internal control by using primers fluorescently labeled with cyanine 5 (Cy5-dUTP) to assess the quality of the DNA. Furthermore, the performance of the PCR was controlled through the addition of a control template to the PapilloCheck assay master mixture. The PCR products were hybridized to HPV type-specific oligonucleotide probes immobilized on the DNA chip and were detected by the binding of a Cy5-dUTP-labeled oligonucleotide probe to the tag sequence. The PapilloCheck assay DNA chip was scanned by the CheckScanner apparatus at wavelengths of 532 nm and 635 nm. Extraction, amplification, hybridization, and orientation controls, as well as type-specific probes, were spotted on each chip fivefold. A sample was considered suitable for evaluation in case all controls (five each) were positive and/or a specimen was positive for at least two spots of any HPV type (i.e., if the signal-to-noise ratio [SNR] was greater than the automatically defined threshold). The PapilloCheck assay is capable of detecting HPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 68, 73, 82, 53, 66, 6, 11, 40, 42, 43, 44/55, and 70. The test cannot distinguish between HPV-55 and HPV-44 due to cross-reaction.
The result of the PapilloCheck assay was initially considered hrHPV positive when genotyping revealed 1 or more of the 17 (probably) hrHPV genotypes described by Munoz et al. (15). For further comparative analysis by the hrHPV GP5+/6+-PCR-EIA, PapilloCheck assay positivity for one or more of the 14 hrHPV genotypes detectable by the GP5+/6+-PCR-EIA was considered. The clinical sensitivity and clinical specificity values of the PapilloCheck assay were compared to those of the GP5+/6+-PCR-EIA by using a noninferiority score test (14). The thresholds used for noninferiority were sensitivity for the detection of ≥CIN3 of at least 90% and specificity for the detection of ≥CIN2 of at least 98% relative to the results of the GP5+/6+-PCR-EIA. These lower bounds were the same as those proposed in recent guidelines that ensure that the negative predictive values are much higher than those of cytology and, at the same time, that there are a limited number of false-positive test results, which result in unnecessary follow-up procedures (14).
To investigate the type-specific agreement for the 14 hrHPV types that are detected by both assays, the typing results were considered to be either concordant (completely identical genotyping results), compatible (both methods detected one or more genotypes in common), or discordant (there was no similarity between the genotypes detected by the two methods). The two-tailed McNemar test was used for mutual comparison of the type-specific positivity rates of both assays. The level of agreement was determined by using the kappa statistic, and the χ2 test was used to compare the distribution of multiple infections between the assays. Except for noninferiority score testing, SPSS (version 15.0) software was used for statistical computations. R software was used for noninferiority score analysis. The level of statistical significance was set at a value of 0.05.
In this study, a series of 1,665 cervical scraping samples collected at the baseline from women participating in the POBASCAM trial were subjected to analysis by the PapilloCheck assay. These included a representative set of 192 scrapings from women with ≥CIN3 detected at a median follow-up time of 5.3 months (range, 0 to 34.6 months) from the time of sample collection and 1,473 samples from women with normal cytological findings without a diagnosis of ≥CIN2 within a median follow-up time of 58.7 months (range, 37.2 to 99.5 months) from the time of sample collection. A valid PapilloCheck test result was obtained for all 192 cases and 1,437 of the 1,473 controls (97.6%; Table Table1).1). The technical specifications of the samples that gave an invalid PapilloCheck test result were as follows: for 24 samples, the result for the sample control (housekeeping gene) was negative; for 2 samples, the result for the PCR control was negative; and for 10 samples, the results for both the sample and the PCR controls were negative. Only the data for samples with a valid PapilloCheck test result were used for further analysis.
Of the cases, 185 (96.4%) scored PapilloCheck assay positive for the 17 (probably) hrHPV types and 2 (1%) scored PapilloCheck assay positive for lrHPV types, whereas 5 (2.6%) scored HPV negative (Table (Table1).1). Of the controls, the PapilloCheck assay found that 53 (3.7%) samples were (probably) hrHPV positive and 22 (1.5%) were lrHPV positive, leaving 1,362 (94.8%) negative samples. Table Table22 shows the PapilloCheck assay results, stratified for cases and controls, in comparison to the GP5+/6+-PCR-EIA findings, both after the inclusion of all 17 (probably) hrHPV types in the PapilloCheck test results and after restriction of the analysis to the 14 hrHPV types (i.e., HPV types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68) that are detectable by the GP5+/6+-PCR-EIA (Table (Table2).2). Altogether, the PapilloCheck assay had a clinical sensitivity for the detection of ≥CIN3 of 96.4% (185/192; 95% confidence interval [CI], 93.7 to 99.7%) and a clinical specificity for the detection of ≥CIN2 of 96.3% (53/1,437; 95% CI, 95.3 to 97.3%) when it was used to detect the 17 (probably) hrHPV types. After restriction of the PapilloCheck assay analysis to the 14 hrHPV types targeted by the GP5+/6+-PCR-EIA, the clinical sensitivity and the clinical specificity values were 95.8% (184/192; 95% CI, 92.8 to 98.8%) and 96.7% (47/1,437; 95% CI, 95.7 to 97.7%), respectively. By comparison, these values were 96.4% (185/192; 95% CI, 93.9 to 98.9%) and 97.7% (33/1437; 95% CI, 96.9 to 98.5%), respectively, for the GP5+/6+-PCR-EIA (Table (Table2).2). The one case sample that was PapilloCheck assay positive after analysis for the 17 types but negative after analysis for the 14 types contained HPV-82. Four case samples tested positive by the GP5+/6+-PCR-EIA but negative by the PapilloCheck assay, whereas three case samples were positive by the PapilloCheck assay (by analysis for 14 types) and negative by the GP5+/6+-PCR-EIA. After repeat testing by the PapilloCheck assay, three of the four PapilloCheck assay-negative and GP5+/6+-PCR-EIA-positive case samples tested positive for the same genotype(s) originally found by the GP5+/6+-PCR-EIA. Of the three cases missed by the GP5+/6+-PCR-EIA but positive by the PapilloCheck assay (by analysis for 14 types), two became positive with type concordance and one became positive with type discordance compared to the results of genotyping by the PapilloCheck assay after repeat testing by GP5+/6+-PCR-EIA following DNA extraction. For the control samples, the PapilloCheck assay analyzing 17 types scored 24 more samples positive than the GP5+/6+-PCR-EIA and the PapilloCheck assay analyzing 14 types scored 18 more samples positive than the GP5+/6+-PCR-EIA. The six samples that were additionally detected by the PapilloCheck assay (by analysis of 17 types) were positive for HPV types different from the 14 detected by the GP5+/6+-PCR-EIA. Of the 18 samples that were additionally detected by the PapilloCheck assay (by analysis of 14 types). Original GPS5+/6+-PCR-EIA values from the crude extracts were elevated but below the cutoff used. Twelve of them were subsequently positive by the GP5+/6+-PCR-EIA with the isolated DNA and all 12 samples had one or more HPV types in common with those found by the PapilloCheck assay (data not shown). Of the six remaining discrepant samples, three were positive for HPV-68, one was positive for HPV-52, one was positive for HPV-56, and one was positive for HPV-59 by the PapilloCheck assay.
We next performed a noninferiority score test to determine whether the clinical sensitivity of the PapilloCheck assay for the detection of ≥CIN3 and the clinical specificity of the PapilloCheck assay for the detection of ≥CIN2 were noninferior to those of the GP5+/6+-PCR-EIA at thresholds of 90% and 98%, respectively. After the inclusion of all 17 (probably) hrHPV types in the analysis, the clinical sensitivity of the PapilloCheck assay was noninferior to that of the GP5+/6+-PCR-EIA (P < 0.0001), but the specificity was inferior to that of the GP5+/6+-PCR-EIA (P = 0.08). When the analysis was restricted to the 14 hrHPV types detected by the GP5+/6+-PCR-EIA, both the clinical sensitivity and the clinical specificity of the PapilloCheck assay were noninferior to those of the GP5+/6+-PCR-EIA (P < 0.0001 and P = 0.007, respectively).
At the analytical level, the overall agreement for the detection of hrHPV (by analysis for 14 types) was 98.2% (1,600/1,629; 95% CI, 97.6 to 98.8%), with the overall kappa value being 0.93. The PapilloCheck assay and GP5+/6+-PCR-EIA reverse line blot genotyping revealed among the samples that tested positive concordant genotyping results for 164 (78%) specimens, compatible results for 41 (20%) specimens, and discordant genotyping results for only 5 specimens (2%) for 1 or more of the 14 GP5+/6+-PCR-EIA hrHPV types. The individual genotyping findings are shown in Table Table3.3. The assays showed good genotyping agreement (kappa value, >0.60) for all types except HPV-68, although the number of samples positive for this HPV type by both assays was low. Even though the PapilloCheck assay detected significantly more HPV type 31, 51, 52, and 59 infections than the GP5+/6+-PCR-EIA (Table (Table3),3), these mainly involved samples harboring multiple hrHPV types (χ2 test, P = 0.02).
In the past, the PapilloCheck assay has been compared with hc2 and the GP5+/6+-PCR-EIA as well as the Linear Array assay (Roche Diagnostics) (7, 11), but these comparisons merely involved the analytical features of the assays. In the present study, we compared the clinical performance characteristics of the PapilloCheck assay with those of the GP5+/6+-PCR-EIA with samples from a cohort of participants originally screened by both cytology and the GP5+/6+-PCR-EIA for the detection of HPV type 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, or 68. When the analysis by the PapilloCheck assay was restricted to the 14 hrHPV types detectable by the GP5+/6+-PCR-EIA, both the clinical sensitivity for ≥CIN3 and the clinical specificity for ≥CIN2 were noninferior to that of the GP5+/6+-PCR-EIA by the use of predetermined thresholds of 90% and 98%, respectively. The inclusion of three additional (probably) hrHPV types (i.e., HPV types 53, 73, and 82) resulted in such a decrease in clinical specificity that the PapilloCheck assay became inferior to the GP5+/6+-PCR-EIA for this parameter. These findings confirm earlier findings that the addition of the ability to detect too many types to the test panel mainly has an adverse impact on the clinical specificity, whereas the gain in clinical sensitivity is only marginal (12, 14).
In recent guidelines for validation of the use of candidate HPV assays for screening purposes, the proposed validation strategy indicates that the candidate test should have a sensitivity for the detection of ≥CIN2 of at least 90% of the sensitivity of hc2, whereas the specificity for the detection of ≥CIN2 should be at least 98% of that of hc2 for women at least 30 years of age (14). In these guidelines, hc2 was chosen as the reference test since that assay, unlike the GP5+/6+-PCR-EIA, is commercially available and therefore widely applicable. In our study, the reference test was the GP5+/6+-PCR-EIA rather than hc2. Like hc2, this assay is clinically validated for use for screening purposes in large clinical trials, and its clinical compatibility with hc2 is further supported by its noninferiority to hc2 in the validation analysis mentioned above (8, 14). This indicates that this assay is a valid reference test for use for validation testing as well. Furthermore, since the GP5+/6+-PCR-EIA is a PCR-based assay with genotyping options, similar to the PapilloCheck assay, it allows comparative analysis at the genotype level. It is noteworthy, however, that recent guidelines also include a requirement for sufficient intralaboratory reproducibility over time and interlaboratory agreement, as determined by the evaluation of at least 500 samples, before a test can be considered suitable for cervical screening purposes (14).
Overall, there was good genotyping agreement between the two assays, although the PapilloCheck assay seemed to detect more HPV type 31, 51, 52, and 59 infections, particularly in individuals with multiple infections. This good analytical agreement is consistent with the findings of Jones et al. (11), although in that study the detection of HPV-35 appeared to be an outlier, being found significantly more often by the GP5+/6+-PCR-EIA than by the PapilloCheck assay. However, in the study of Jones et al. (11), enzyme immunoassay instead of reverse line blot analysis was used for the genotyping of the GP5+/6+-PCR products. The controls that were additionally detected by the PapilloCheck assay most likely harbored lower viral loads in the crude sample extract that originally remained below the assay detection threshold of the GP5+/6+-PCR-EIA. The three discrepant control samples for which the PapilloCheck assay identified HPV-68 most likely contained an HPV-68 variant that was less well detected by the primer combination used in the GP5+/6+-PCR, as has been described by Söderlund-Strand et al. (21) and Schmitt et al. (19). Nevertheless, our study indicates that possible minor differences at the analytical level have no major impact on the clinical performance characteristics of the two assays.
In conclusion, this study showed that when the 14 hrHPV types detectable by the GP5+/6+-PCR-EIA are considered, the PapilloCheck assay is clinically compatible with the GP5+/6+-PCR-EIA for the detection of ≥CIN3.
The PapilloCheck test kits were kindly provided by Greiner Bio-One.
Published ahead of print on 30 December 2009.