The 2010 Global Proficiency Study of Human Papillomavirus Genotyping in Vaccinology

doi:10.1128/JCM.00840-12

J Clin Microbiol. 2012 July; 50(7): 2289–2298.

PMCID: PMC3405610

The 2010 Global Proficiency Study of Human Papillomavirus Genotyping in Vaccinology

Carina Eklund,^a^b Ola Forslund,^a Keng-Ling Wallin,^c Tiequn Zhou,^d and Joakim Dillner^a^b, for the WHO Human Papillomavirus Laboratory Network

^aWHO HPV LabNet Global Reference Laboratory, Departments of Clinical and Medical Microbiology, Laboratory Medicine Skåne, and Lund University, Malmö, Sweden

^bDepartments of Laboratory Medicine, Medical Epidemiology & Biostatistics, Karolinska Institute and Hospital, Stockholm, Sweden

^cEQUALIS AB, Uppsala, Sweden

^dQuality, Safety and Standards Team, Immunizations, Vaccines and Biologicals Department, Family and Community Health, World Health Organization, Geneva, Switzerland

Corresponding author.

Address correspondence to Joakim Dillner, Email: Joakim.dillner/at/med.lu.se.

Received March 28, 2012; Revisions requested April 10, 2012; Accepted April 14, 2012.

This article has been cited by other articles in PMC.

Abstract

Accurate and internationally comparable human papillomavirus (HPV) DNA genotyping is essential both for evaluation of HPV vaccines and for effective monitoring and implementation of vaccination programs. The World Health Organization (WHO) HPV Laboratory Network (LabNet) regularly issues international proficiency studies. The 2010 HPV genotyping proficiency panel for HPV vaccinology contained 43 coded samples composed of purified plasmids of 16 HPV types (HPV types 6, 11, 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68a and 68b) and 3 coded extraction controls. Proficient typing was defined as detection in both single and multiple infections of 50 international units (IU) of HPV type 16 (HPV-16) and HPV-18 DNA and 500 genome equivalents (GE) for the other 14 HPV types. Ninety-eight laboratories worldwide submitted a total of 132 data sets. Twenty-four different HPV genotyping assay methods were used, with Linear Array being the most commonly used. Other major assays used were a line blot assay (Inno-LiPa), CLART, type-specific real-time PCR, PCR Luminex, and different microarray assays. Altogether, 72 data sets were proficient for detection of more than 1 type, and only 26 data sets proficiently detected all 16 HPV types. The major oncogenic HPV types, 16 and 18, were proficiently detected in 95.0% (114/120) and 87.0% (94/108) of data sets, respectively. Forty-six data sets reported multiple false-positive results and were considered nonproficient. A trend toward increased sensitivity of assays was seen for the 41 laboratories that participated in both 2008 and 2010. In conclusion, continued global proficiency studies will be required for establishing comparable and reliable HPV genotyping services for vaccinology worldwide.

INTRODUCTION

Cervical cancer is the second most common type of cancer among women worldwide, with human papillomavirus (HPV) infection linked to more than 99% of cervical cancers (3, 37). The most important high-risk types, HPV type 16 (HPV-16) and HPV-18, account for about 70% of all invasive cervical cancers worldwide (27).

An accurate and internationally comparable HPV DNA detection and genotyping methodology is an essential component both in the evaluation of HPV vaccines and in the effective implementation and monitoring of HPV vaccination programs. Genotyping assays used today differ in their analytical performance with regard to type-specific sensitivity and specificity (15). Several studies have compared different HPV typing assays using various clinical samples to assess their performance (7, 17, 24). However, in addition, evaluation of assay performance needs to be performed in a standardized manner, where different assay performances can be evaluated and results can be compared against a known and accepted standard over time (15).

In 2005, the World Health Organization (WHO) initiated the establishment of the global HPV Laboratory Network (LabNet) with the objective to facilitate the development and implementation of HPV vaccines by improving and standardizing the quality of HPV laboratory services used for HPV surveillance and HPV vaccination impact monitoring. The main activities within the HPV LabNet include harmonization and standardization of laboratory procedures by the development of internationally comparable quality assurance methods, international standards, and reference reagents and standard operating procedures (SOPs) for vaccinology (8, 9, 38).

Regularly issued global proficiency studies are essential for establishing comparable and reliable laboratory services. A number of international proficiency panels for quality assurance of laboratory testing are being conducted widely for a number of infectious agents. For example, the WHO measles and rubella laboratory networks have been distributing proficiency panels worldwide yearly since 2001, with the purpose to monitor the performance of laboratories and assay methodology over time (31). As there is no natural source of biological material that could be used to generate type-specific HPV international standards (ISs), recombinant HPV DNA plasmids were used to establish ISs of HPV-16 and HPV-18 DNA in 2008 with an assigned potency in international units (IU) (39). In 2008, the WHO HPV LabNet conducted a proficiency study based on HPV DNA plasmids containing the genomes from 14 oncogenic HPV types and 2 benign HPV types and open for participation to laboratories worldwide (6). That study demonstrated that it is possible to perform global proficiency studies with unitage traceable to ISs based on plasmid DNA and that such studies can provide an overview of the status of the HPV detection and typing methodology worldwide.

The international WHO proficiency study described herein concerns vaccinology. It should be realized that the proficiency reported cannot be translated into proficiency for cervical screening, as the latter demands HPV testing to be informative about the presence of cervical (pre)cancerous lesions and as such has different (analytical) requirements. This report was based on a proficiency panel composed of the same HPV DNA plasmid material used in 2008, with the amount of DNA titrated in amounts traceable to the IS. The use of the same panel material allowed a reproducible, standardized evaluation of assay sensitivity over time. Specificity was defined as absence of incorrect typing. The sample preprocessing was evaluated with extraction controls of cervical cancer cell lines. The panel was distributed to 105 laboratories worldwide and analyzed using a range of HPV DNA typing assays in a blinded manner. We report the results in terms of the ability of the participating laboratories to correctly identify the HPV types, grouped by the methods used, assess the analytical sensitivity for the detection of the HPV types included in the study, and report on the test results on the comparison of the panel for the years 2008 and 2010.

MATERIALS AND METHODS

Source of panel material.

Complete genomes of HPV cloned into plasmid vectors were provided to Lund University by the respective proprietors with written approval for use in this proficiency panel: Ethel-Michele de Villiers (HPV types 6, 11, 16, 18, and 45), Gérard Orth (HPV types 33, 39, and 66), Ola Forslund (HPV-68a L1), Elisabeth Schwarz (HPV-68b), Saul Silverstein (HPV type 51), Attila Lörincz (HPV types 31, 35, and 56), Wayne Lancaster (HPV type 52), and Toshihiko Matsukura (HPV types 58 and 59). The agreements allowed distribution of the plasmids only for the performance of this WHO proficiency study.

The HPV genomes are cloned into different cloning vectors: HPV-6 in pBR322 at position 4724 in the HPV genome, HPV-11 in pGEM4Z at position 4781, HPV-16 in pBR322 at position 6152, HPV-18 in pBR322 at position 2440, HPV-31 in pT713 at position 3362, HPV-33 in pBR322 at position 2797, HPV-35 in two fragments (positions 5012 to 956 and 956 to 5012) in pT713, HPV-39 in pGEM4z at position 1714, HPV-45 in pGEM4Z at position 75, HPV-51 in pGEM4z at position 4511, HPV-52 in pUC19 at position 7559, HPV-56 in pT713 at position 5521, HPV-58 in pGEM4Z at position 1158, HPV-59 in pUC9 at position 69, HPV-66 in pBR322 at position 7484, HPV-68a L1 from a clinical sample cloned in pCR-Script, and HPV68b (ME180) of about 7 kb containing the L1, L2, E1, E2, E4, E5, E6, and E7 genes and an incomplete E2 gene in pCR4-TOPO.

The nucleic acid sequences for each of these HPV genomes have been reported previously and are available in GenBank with the following accession numbers; HPV-6, X00203; HPV-11, M14119; HPV-16, K02718; HPV-18, X05015; HPV-31, J04353; HPV-33, M12732; HPV-35, M74117; HPV-39, M62849; HPV-45, X74479; HPV-51, M62877; HPV-52, X74481; HPV-56, X74483; HPV-58, D90400; HPV-59, X77858; HPV-66, U31794, HPV-68a, X67161; and HPV-68b, FR751039.

Panel composition and production.

The plasmids were prepared and characterized as previously described (36), with the following changes: HPV-39 was originally cloned into the L1 gene at the binding site for one of the PGMY primers and was therefore recloned so that the vector (pGEM4z) is now positioned in the E1 gene (position 1714). A plasmid for HPV-68b (ME180) of about 7 kb containing the L1, L2, E1, E2, E4, E5, E6, and E7 genes and an incomplete E2 gene was added to the panel. Purified plasmids containing cloned genomic DNAs for HPV types 6, 11, 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68a, and 68b were prepared to make up the 43 different panel samples by diluting the HPV recombinant DNA plasmid stock solution in TE (Tris-EDTA) buffer in the background of human placental DNA, as previously described (36). Table 1 summarizes the composition of the panel. The different amounts of plasmids (5 to 500 GE or IU) were chosen to reflect the lower spectrum of the amount of virus that would typically be present in clinical samples (28). After production of each of the 43 reference samples, the preparation was dispensed in 100-μl volumes in 1.5-ml siliconized vials. The vials were labeled WHO HPV DNA 2010 and were randomly assigned numbers from 1 through 43. The panels were stored at −20°C before shipment to participating laboratories. Participants were instructed to perform HPV typing according to their standard methods using their standard sample input volume.

Table 1

HPV DNA proficiency panel composition and HPV testing results

Two different cell lines were used as controls of the extraction process in participating laboratories. The HPV-negative epithelial cell line C33A, derived from a human cervical carcinoma, and the HPV-16-positive epithelial cell line SiHa, derived from a squamous cell carcinoma, were purchased from the American Type Culture Collection and cultured in Dulbecco's modified Eagle medium (11960; Gibco). The cells were diluted in PreservCyt transport medium (0234004; Cytyc Corporation) to concentrations of 5 and 500 SiHa cells/μl in a background of 5,000 C33A cells/μl. One sample contained only the background C33A cells. Two hundred microliters of each preparation was dispensed in 1.5-ml vials and labeled WHO HPV DNA A, B, or C.

Before distribution of the WHO HPV DNA proficiency panel, the samples were tested (blinded) by the WHO HPV LabNet Global Reference Laboratory (GRL) in Sweden and one other laboratory, namely, the German Cancer Research Center (DKFZ) in Heidelberg, Germany (Michael Pawlita).

Technologies used for initial characterization of the panel. (i) GRL in Sweden.

Three independent experiments testing each sample in duplicate were performed. Five microliters of panel sample DNA was used for modified general primer (MGP) PCR as previously described (29). Ten microliters PCR products was analyzed by multiplex genotyping using a Luminex-based assay that can distinguish HPV types 6, 11, 16, 18, 26, 30, 31, 33, 35, 39, 40, 42, 43, 45, 51, 52, 53, 54, 56, 58, 59, 61, 62, 66, 67, 68a, 68b, 69, 70, 73, 81, 82, 86, 89, 90, 91, and 114, as described earlier (25, 26).

Appropriate negative and positive controls were used to monitor the performance of the method. DNA was extracted from extraction controls A, B, and C using a QIAamp DNA mini- and blood kit (Qiagen) according to the manufacturer's instructions.

(ii) DKFZ.

A 10-μl DNA sample was tested by the BSGP5+/6+-PCR/MGP assay, as previously described (26). The PCR products were analyzed using bead-based multiplex genotyping as described previously (25). HPV types 6, 11, 16, 18, 26, 30, 31, 33, 35, 39, 42, 43, 44, 45, 51, 52, 53, 56, 58, 59, 66, 67, 68a, 68b, 69, 70, 73, and 82 can be distinguished by this method. All samples were tested for human DNA with PCR primers targeting a part of the β-globin gene and a bead-coupled β-globin gene-specific probe used in the genotyping assay.

Organization of the study.

A call for participation in the study was advertised on the WHO public website. Laboratories that are or will be involved in HPV surveillance and/or vaccine development were particularly welcome. The panels were prepared by the WHO HPV LabNet GRL in Sweden and were distributed by EQUALIS, the quality assurance company in Sweden, at ambient temperature to 105 laboratories worldwide. The number of participating laboratories according to WHO regions are as follows: America region, 23 laboratories; Africa region, 1 laboratory; Eastern Mediterranean region, 5 laboratories; European region, 49 laboratories; South East Asia region, 9 laboratories; and Western Pacific region, 18 laboratories. The package also included a letter of instruction, a form for reporting the results of the testing of the panel, as well as technical information on the procedures to be performed. Laboratories were asked to submit the results of the tests performed to EQUALIS online within 4 weeks of receipt of the specimens. In registering for the proficiency study, the participating laboratory agreed to assign the right to publish the data to the WHO; it also agreed that only coded results from all participating laboratories, grouped by methods performed, would be presented.

All results submitted to EQUALIS were coded and analyzed anonymously by the GRL in Sweden. Data sets generated were designated numerically from 1 through 132. Individual results of the proficiency study were disclosed only to the participating laboratory that generated the data.

HPV technologies used by study participants.

The different HPV typing methods that were used to generate results for the WHO LabNet proficiency study to detect HPV DNA (1, 2, 4, 5, 8, 10–14, 16, 20–22, 25, 26, 29, 30, 32–35) are summarized in Table 2.

Table 2

Proficiency of detecting HPV types tested for, by assay^a

Data analysis.

Criteria used for considering a data set proficient for HPV vaccinology were the following: (i) detection of at least 50 IU per 5 μl of HPV-16 and HPV-18 in both single and multiple HPV infections, (ii) detection of at least 500 GE per 5 μl of the other HPV types included in both single and multiple infections, and (iii) at most, one false-positive result. These criteria were arrived at by a consensus opinion of international experts participating in an international WHO workshop in Geneva, Switzerland, in 2008 (9) and were based on consideration of which performance requirements are required and are realistic. A higher requirement for HPV-16 and -18 was considered essential, because of the pivotal role of these HPV types in causing cervical cancer.

Four data sets reporting results only as “high-risk” or “low-risk” HPV were not included in the overall performance analyses (one such data set was generated using the Hybrid Capture 2 assay [Qiagen], one data set was generated using an in-house assay, and two data sets were generated using the Cobas 4800 system [Roche]).

RESULTS

Validation of HPV proficiency panel.

The results from the initial panel validation at the GRL in Sweden and at DKFZ included qualitative characterization of HPV and human genomic DNAs. Both of these laboratories used Luminex-based assays with modified GP5+/6+ primers. No false-positive HPV type was detected in the samples in any of the reference laboratories. Most HPV types were detected by both laboratories in the lowest concentration included in the panel. Exceptions were HPV-31, -33, -39, and -58, when present together with other plasmids that could be detected by only one of these laboratories. Both reference laboratories detected HPV-16 DNA in the DNA extraction control containing SiHa cells and had negative results in the negative control for DNA extraction (C33A cells). The results from the reference laboratory evaluation advised that the panel performed as expected, and the panel was then distributed to the participating laboratories worldwide.

Panel distribution and response.

Ninety-eight of 105 participating laboratories, including the 2 laboratories that conducted the panel validation, submitted 132 data sets according to the timeline (Table 2). Four data sets were generated using assays that did not discriminate specific HPV types and were therefore not included in the overall type-specific analyses presented here. Some participating laboratories did not perform tests for typing of all HPV types included in the proficiency panel. Therefore, the denominator for the number of test results included in the analyses varies for the different HPV types.

In 73 data sets, the results were obtained using commercially available tests. The most commonly used assay was Linear Array (Roche), which was used to generate 17 data sets. Other widely used assays included Inno-LiPA (Innogenetics), CLART HPV 2/3 (Genomica), PGMY-CHUV (Centre Hospitalier Universitaire Vaudois), and other in-house line blot, in-house type-specific PCR, Luminex, and microarray-based assays (Table 2). Participating laboratories included public health laboratories, research laboratories, diagnostic kit manufacturers, and vaccine companies. According to the survey, the number of samples analyzed for HPV typing per laboratory varied from 60 to 100,000 per year, with approximately 52% of the laboratories performing less than 2,000 HPV typing tests per year, about 35% performing between 2,000 and 10,000 assays per year, and 12% of the laboratories performing more than 10,000 HPV genotyping assays yearly.

Performance of HPV assays and participating laboratories.

Participating laboratories were requested to perform testing using their standard protocols. Accordingly, the input volume of the DNA panel varied between 1 μl and 50 μl between laboratories. Data are presented by the lowest category of the concentration (5, 50, or 500 GE or IU) proven to be detectable. For example, a lab using a 2-μl input instead of a 5-μl input that does detect 2 GE is considered to be able to detect 5 GE. HPV-16 and -18 were included as single plasmids at the highest concentration of 10 IU/μl and could be correctly detected in 95% and 87% of the data sets, respectively. The samples containing single plasmids at a concentration of 100 GE/μl of HPV-6, -11, -33, -58, and -66 were correctly identified, without false-positive types detected, in more than 95% of the data sets (Table 1). HPV-39 and -68b were correctly identified in less than 80% of the data sets. HPV-68a cannot be detected by Linear Array and other PGMY-based assays because of several primer mismatches. HPV-68a was correctly identified by only 36.8% of the laboratories. In the samples containing multiple HPV types, between 44% and 78% of the data sets could correctly identify the types. The negative-control sample containing only human genomic DNA was correctly identified as negative by 128 of 132 data sets.

The proficiency of detecting HPV types by assay (restricted to data sets testing for more than 2 HPV types) is shown in Table 2. Twenty-six data sets were 100% proficient (detecting at least 50 IU of HPV-16 and HPV-18 in 5 μl and 500 GE in 5 μl of the other HPV types tested for when also present together with other HPV types), as they did not have more than one false-positive result. As the Linear Array assays used a large (50-μl) input volume in some laboratories, these Linear Array data sets did not test for the presence of amounts below 50 IU of HPV-16 and HPV-18 in 5 μl and 500 genome equivalents of the other HPV types in 5 μl. For the commercial assay Papillocheck, the panel did not evaluate the ability to detect HPV-18 since the HPV-18 plasmid included in the panel is cloned in the region targeted by this assay (E1).

Two different microarray assays, Papillocheck (Greiner Bio-one) and EASYChip (King Car), were the commercial tests that had the highest number of proficient results (100%). About half of the data sets generated by Linear Array were 100% proficient. Several in-house assays based on general primer PCR followed by hybridization (PGMY-CHUV) or Luminex were also 100% proficient.

To be considered proficient in this study, no more than one false-positive sample per data set was acceptable. The number of false-positive HPV types detected per data set is shown in Table 3. Seventy-one of the 132 data sets did not have any false-positive results, whereas 26 data sets reported more than 3 false-positive results. Among these, 4 data sets reported false-positive HPV types in more than 25 samples. More than one false-positive result was generated by the commercial tests InnoLiPA, CLART HPV 2/3, and Linear Array in 75%, 50%, and 35% of the data sets, respectively. Several in-house and commercial assays that were performed by only a few laboratories reported no false-positive results at all.

Table 3

False-positive HPV types detected, by assay

The lowest numbers of GE or IU of each HPV type that was included in the panel and that was detected in both single and multiple infections by different assays are shown in Table 4. HPV-11, -16, -18, -33, -52, and -66 were the types detected at the lowest concentration in most data sets. Only 3 data sets could not detect the highest concentration of HPV-16. In contrast, for HPV-39, HPV-59, and HPV-56, there were 41, 37, and 32 data sets, respectively, that could not detect these viruses in the highest concentration (Table 4).

Table 4

HPV IU/GE detection per 5 μl in both single and multiple infections^a

Three additional samples (A, B, C) were used to evaluate the DNA extraction step before the HPV testing and typing. Two of the samples contained cells from the cervical cancer cell line SiHa mixed with the HPV-negative cancer cell line C33A in different amounts, and one sample with only C33A cells was the negative control. We did not observe any obvious difference in performance between different extraction methods (data not shown). In the sample containing 2,500 cells/5 μl of the cervical cancer cell line SiHa, HPV-16 was correctly identified by 83% of the data sets. Four data sets reported false-positive HPV types in this sample. The negative control containing only C33A cells was correctly reported to be negative by only 83% of the laboratories (Table 1).

Comparison of results for laboratories that participated in the proficiency study in both 2008 and 2010.

Forty-one laboratories analyzed the proficiency panels in both 2008 and 2010. Some of the laboratories used the same tests in both years, whereas some laboratories had changed at least one of the tests used. Percent proficiency for both years and in comparison with the results from all data sets submitted in 2010 is shown in Table 5. Laboratories that used the same assay in both years were 27% proficient in 2008, whereas they were 30% proficient in 2010. There was a definite trend toward increased sensitivity of assays; e.g., 50 IU of HPV-16 could be detected by all (100%) laboratories in 2010 and by 86% in 2008 (data not shown). However, the increase in sensitivity for several laboratories is accompanied by increased amounts of false-positive results, resulting in nonproficiency (Table 6).

Table 5

Proficiency of detecting HPV types by laboratories that participated in both 2008 and 2010 proficiency studies in comparison with all data sets submitted in 2010^a

Table 6

Number of false-positive HPV results detected per data set reported by laboratories participating in both 2008 and 2010 proficiency studies in comparison with all data sets submitted in 2010

DISCUSSION

We report on a reproducible, internationally comparable quality assurance methodology traceable to ISs. The methodology for evaluation of laboratory performance needs to be standardized, in order to enable accurate comparison of the methodologies used in laboratories worldwide.

The current study has established that repeated issuing of international proficiency panels containing known amounts of virus plasmids with unitage traceable to ISs can be used to follow the development of the HPV typing methodologies for vaccinology that are being used globally and how robust they are when performed in different laboratories.

Overall, a majority of HPV DNA typing methodologies used by laboratories participating in this study had a proficient performance according to the established criteria. However, some limitations were revealed.

The 2008 study findings that there were systematic differences in the sensitivity to detect different HPV types remained in 2010. For example, HPV-16, HPV-11, and HPV-18 were still the types detected at the smallest amount in most data sets (only 3, 9, and 11 data sets, respectively, could not detect 500 IU/5 μl), whereas HPV-39, HPV-59, and HPV-56 could not be detected in the 500-GE/5-μl amount by 41, 37, and 32 data sets, respectively. This suggests that many surveys of circulating HPV types systematically underestimate the prevalence of HPV-39, -56, and -59 compared to that of HPV-16 and -18. As also found in 2008, HPV-52, -56, and -59 were the types most difficult to detect.

Correct typing of samples containing multiple HPV types was reported in 44% to 78% of the data sets, in comparison to an average of 86% when only 1 HPV type was present in the sample. A lower sensitivity in samples with multiple types was also seen in the 2008 study. The underestimation of the prevalence of multiple infections will introduce a systematic detection bias in epidemiological studies, with detectability being dependent on determinants of HPV acquisition. Some high-risk HPV types will thus be more difficult to detect in patients in high-risk groups, because of a higher likelihood of multiple HPV infections.

There was a rather large amount of false-positive results reported, with only 71/132 (54%) of the data sets being 100% specific. This is a small, but noteworthy, improvement compared to the results in 2008, when only 42% (34 of 80) of the data sets were 100% specific.

The proficiency panel contained only 2 entirely HPV-negative samples. The study was designed to evaluate HPV typing, and we considered that in this context specificity should be measured primarily as absence of detection of a specific HPV type when other HPV types are also present. Thus, for each HPV type evaluated, at least 38 negative samples were included in the panel, and 1 false-positive result thus equals >97% specificity.

We searched the data sets for patterns of consistent false positivity for any specific sample in the panel. The false-positive results appeared to be essentially randomly distributed among the samples, indicating that the problem with false positives is usually not related to a property of the assay itself (e.g., cross-reactivity) but rather is related to the laboratory conditions of use (e.g., contamination).

A systematic false positivity was found for the samples that contained the HPV-58 plasmid, where 15 data sets also detected HPV-52 in at least one of the HPV-58-containing samples. This could be related to the fact that both the Linear Array and InnoLiPA assays state that these tests cannot exclude HPV-52 detection in samples that contain HPV-58. Most of the HPV-52 detections in the HPV-58-positive samples were generated using the SPF10 primers used in InnoLiPA, but there were also other assays, including HPV-52 type-specific PCRs. As HPV-52 and HPV-58 are closely related viruses, it is conceivable that several assays could have problems with distinguishing these HPV types. However, it should also be considered whether these samples could have been contaminated in the proficiency panel itself. There were no less than 94 data sets from laboratories proficient to detect HPV-52 in the lowest dilution that did not report this false HPV-52 positivity in these samples, and several of them used the same assays as those reporting the false HPV-52 positivity, suggesting that a general proficiency panel contamination is unlikely as an explanation.

Some needs for improvement of the proficiency panel itself were identified by this study. The commercial test Papillocheck, used by 4 laboratories, uses primers directed to the E1 gene. Since the plasmid used for HPV-18 is cloned at one of the primer binding sites in E1, this assay cannot detect the HPV-18 plasmid and was considered to have not tested for HPV-18 in the study. The plasmid used to test for HPV-68a was not full length but contained only the L1 gene. We noted in 2008 that Linear Array and all other PGMY-based assays that are indeed directed against L1 could not detect the HPV-68a plasmid. In this new panel, a plasmid containing HPV-68b was included in addition to HPV-68a (18, 23). All data sets reporting usage of primers directed to genes other than L1 or that used the PGMY primers were considered to have not tested for HPV-68a in this study. Accordingly, only 61 data sets could be analyzed for detection of HPV-68a. Still, only 17 of these laboratories (28%) could detect HPV-68a. In order to allow detection systems with targets outside L1, full-length genomes of HPV-68a will be included in the next panel.

The most commonly used commercial assay, Linear Array, used to generate 17 data sets, cannot exclude HPV-52 when the sample is positive for HPV-33, HPV-35, or HPV-58. In the 2008 study, 4/15 laboratories scored all samples with multiple infections containing HPV-52 as negative for HPV-52. In 2010, no laboratory scored HPV-52 as negative in multiple infections containing HPV-52, and all laboratories using Linear Array were proficient in detecting HPV-52. Six data sets generated using Linear Array reported between 2 and 10 false-positive results and were considered not proficient. Among the 31 total false-positive results submitted for the 17 data sets using Linear Array, 11 were false positive for HPV-66. Ten of these 11 false-positive detections were in samples that contained HPV-56. This confirms the observation already made in 2008 that the Linear Array assay is prone to false detection of HPV-66 in HPV-56-positive samples.

For the commercial test InnoLiPA, 9 out of 12 data sets reported between 2 and 8 false-positive results. Fifteen out of the 42 false-positive results reported were for HPV-52 detection in samples with HPV-58 plasmids, as described above, and four data sets detected HPV-52 in samples that contained HPV-68b. The other false-positive results appeared to be randomly distributed among the samples and were always different for the different laboratories.

Four of eight laboratories using the assay CLART HPV 2/3 submitted data sets with between 2 and 4 false-positive results. This is a major improvement compared to the study results in 2008, when 3 laboratories using this assay reported 7, 17, and 21 false-positive results, respectively, with some having more than 3 false positives in each sample. This indicates that the previous problem with low specificity is not related to the assay kit itself, and there are also examples of several laboratories that had completely proficient results using this assay.

The line blot assay PGMY-CHUV is described in the WHO HPV laboratory manual (36). The assay was developed within the WHO HPV LabNet (9) in order to provide an inexpensive assay that would be independent of any specific commercial vendor. The 6 different laboratories in 4 different continents that had used this assay generally had good results, with no false-positive results and 4/6 laboratories being fully proficient, supporting the suggestion that this assay is suitable for standardization and technology transfer.

As was also found in our previous study (6), differences in performance were much larger between laboratories than between different types of assays. Proficiency panel testing is thus particularly useful to stimulate a learning process for improved performance in laboratories.

Three samples were included in the panel to evaluate the DNA extraction step before the HPV testing and typing. These contained cells from the cervical cancer cell line SiHa in a background of the HPV-negative cell line C33A to mimic a clinical sample. SiHa cells have about 1 copy of HPV-16 per cell, and HPV-16 was correctly identified in samples with 2,500 cells/5 μl in 83% of the data sets. This is a major improvement compared to the results obtained in 2008, when only one-third of the data sets could detect 2,000 IU of HPV-16/5 μl. In the sample containing only the HPV-negative cell line, 12 data sets reported false-positive results, and in total, 21 false-positive results were reported in the 3 extraction samples. This suggests that, for a noteworthy minority of laboratories, contamination in the DNA extraction step is an issue.

The HPV LabNet has chosen to perform proficiency testing using a panel of HPV plasmids since this material can be used to generate exactly reproducible panels with defined content in quantities that can be distributed to hundreds of laboratories over many years. The use of clinical samples in proficiency panels does not allow the same reproducibility over time. To assess the additional steps in the laboratory detection process that are not evaluated by the current proficiency panel, e.g., evaluation of the sampling technique, handling, and storage and for the presence of PCR-inhibiting substances, the HPV LabNet instead performs quality control by a confirmatory testing scheme. Participating laboratories annually submit a part of their clinical samples tested to a higher-level reference laboratory for retesting (5).

This was the second HPV DNA proficiency panel issued by HPV LabNet that was open for testing by participants worldwide. The number of participating laboratories almost doubled, from 54 laboratories in 2008 to 98 laboratories in 2010. This increased participation in the study shows that many laboratories are interested in quality assurance for their assay methodologies and laboratory performance. Comparing the results of the laboratories that tested both the 2008 and 2010 WHO HPV DNA proficiency panels, we observed only marginal overall improvements. Among laboratories that used the same assay in both years, 27% were proficient in 2008, whereas 30% were proficient in 2010. However, there are several noteworthy examples of laboratories that achieved major improvements. We also saw a strong trend toward increased sensitivity of assays. For example, among the laboratories using the same assay in 2008 and 2010, 50 IU of HPV-16 could be detected by all (100%) laboratories in 2010, whereas 86% of laboratories could detect 50 IU of HPV-16 in 2008. However, for several laboratories, the increased sensitivity was accompanied by increased amounts of false-positive results, resulting in nonproficiency. We suggest that recommendations for HPV laboratory testing include an increased emphasis on the use of negative controls in the assays. Furthermore, we suggest that the requirements for proficiency in future proficiency panels announce at the outset that proficiency requires no false positives at all.

The demands on sensitivity of HPV typing assays vary depending on the purpose of the testing. The WHO HPV LabNet proficiency panels are designed to evaluate the performance of HPV typing tests used in HPV vaccinology and HPV surveillance. In vaccinology, high analytical sensitivity is needed, as failure to detect prevalent infections at trial entry may result in false vaccine failures in vaccination trials. It should be noted that the HPV tests used in cervical cancer screening programs have different requirements for evaluation, since for that purpose, only HPV infections associated with high-grade cervical intraepithelial neoplasia or cancer and not those transient HPV infections that do not give rise to clinically meaningful disease are relevant. Since the latter are characterized by low viral loads, HPV screening assays do not have demands on analytical sensitivity that are as high (19).

In conclusion, we find that the use of global HPV DNA typing proficiency panels for validating different HPV DNA tests and laboratories promotes the comparability of data generated from different laboratories worldwide. Regularly issued global HPV DNA typing proficiency panels that allow comparison of global results over time will be required for the continuing work toward international standardization and quality improvement of HPV DNA typing results worldwide.

ACKNOWLEDGMENTS

Members of the WHO HPV LabNet are as follows: A. C. Bharti, Division of Molecular Oncology, Institute of Cytology and Preventive Oncology, Nodia, India; J. Dillner, Department of Medical Microbiology, Lund University, Malmo, Sweden; E. Ennaifer-Jerbi, Tunis Pasteur Institute, Tunis, Tunisia; S. Garland, Department of Microbiology and Infectious Disease, Royal Women's Hospital, Carlton, Australia; I. Kukimoto, Pathogen Genomics Center, National Institute of Infectious Diseases, Tokyo, Japan; J. Ngamkham, National Cancer Institute, Bangkok, Thailand; A. M. Picconi, National Institute of Infectious Diseases, Buenos Aires, Argentina; R. Sahli, Institut de Microbiologie, CHUV, Lausanne, Switzerland; E. R. Unger, Centers for Disease Control and Prevention, Atlanta, GA; and A. L. Williamson, Institute of Infectious Disease and Molecular Medicine, University of Cape Town, Cape Town, South Africa.

We thank Michael Pawlita of the German Cancer Research Center (DKFZ), which acted as an external validation laboratory in the evaluation of the proficiency panel. We also thank all the 107 participants worldwide that participated in the proficiency study: A. Giri, A. Picconi, A. Severini, I. Gòrska-Flipot, F. Coutlee, G. I. Sanchez, M. Molano, W. McLaughlin, J. Gibson, C. L. Aldrich, B. Bryant, S. Y. Lee, P. Eder, E. R. Unger, M. Nappi, R. Peterson, N. B. Kiviat, L. Wang, J. Sebastian, S. Nishtar, C. Wheeler, M. Guo, F. R. Pastrana, E. Ennaifer-Jerbi, R. Hamkar, M. Farzami, S. M. Samiee, S. A. Nadji, C. Monceyron Jonassen, P. Karakitsos, P. V. Constantoulakis, R. Tachezy, L. Struijk, A. K. Lie, M. Neugebauer, M. Benczik, R. Sahli, P. Snijders, U. Gyllensten, S. Beddows, B. Johansson, M. L. Villahermosa, N. Houard, A. Pista, F. Carozzi, J. Bonde, C. Clavel, E. Andersson, C. Ornelas, J. Dillner, O. Forslund, K. Cuschieri, M. Pawlita, M. Schleichert, S. Zidovec Lepej, C. Gurkan, J. Hariri, E. Auvinen, P. Halfon, M. Favre, P. Soussan, C. Mougin, L. Barzon, M. L. Tornesello, L. Giovannelli, A. Del Mistro, A. Gillio Tos, R. G. Ursu, M. Poljak, S. Perez Castro, M. de Oña Navarro, M. A. Rodríguez-Iglesias, H. Enroth, M. N. C. de Koning, P. Neophytou, J. P. A. Baak, H. J. Boot, C. Saldanha, V. Böhm, J. Ngamkham, U. Utaipat, A. C. Bharti, I. Goud Kalal, A. Priya, D. Saranath, R. K. Vangala, P. Gupta, S. Hartini, Y. E. Wong, D. H. Byoung, Y. S. Park, T. J. Kim, H. Park, S. S. Kim, S. C. Kim, T. Perris, S. Tabrizi, I. Kukimoto, Y. Fujii, Y. L. Qiao, S. T. H. Lo, L. C.H. Tzang, S. Y. Lam, M. Fang, C.-J. Chen, T. C. Su, and A. L. Williamson.

This work was supported by the WHO via a project funded by the Bill and Melinda Gates Foundation.

Tiequn Zhou is a staff member of the World Health Organization. Tiequn Zhou alone is responsible for the views expressed in this publication, and they do not necessarily represent the decisions, policy, or views of the World Health Organization.

Joakim Dillner has acted as a consultant for and received research grants from Merck/SPMSD, a manufacturer of HPV vaccines. Carina Eklund, Keng-Ling Wallin, Ola Forslund, and Tiequn Zhou have disclosed no potential conflicts of interests.

Footnotes

Published ahead of print 25 April 2012

REFERENCES

1. Baay MF, et al. 1996. Comprehensive study of several general and type-specific primer pairs for detection of human papillomavirus DNA by PCR in paraffin-embedded cervical carcinomas. J. Clin. Microbiol. 34:745–747. [PMC free article] [PubMed]

2. Bernard HU, et al. 1994. Identification and assessment of known and novel human papillomaviruses by polymerase chain reaction amplification, restriction fragment length polymorphisms, nucleotide sequence, and phylogenetic algorithms. J. Infect. Dis. 170:1077–1085. [PubMed]

3. Bosch FX, Lorincz A, Munoz N, Meijer CJ, Shah KV. 2002. The causal relation between human papillomavirus and cervical cancer. J. Clin. Pathol. 55:244–265. [PMC free article] [PubMed]

4. Chouhy D, et al. 2006. Development and evaluation of a colorimetric PCR system for the detection and typing of human papillomaviruses. Int. J. Mol. Med. 18:995–1003. [PubMed]

5. Coutlee F, et al. 2007. Confirmatory real-time PCR assay for human papillomavirus (HPV) type 52 infection in anogenital specimens screened for HPV infection with the Linear Array HPV genotyping test. J. Clin. Microbiol. 45:3821–3823. [PMC free article] [PubMed]

6. Eklund C, Zhou T, Dillner J. 2010. Global proficiency study of human papillomavirus genotyping. J. Clin. Microbiol. 48:4147–4155. [PMC free article] [PubMed]

7. Else EA, et al. 2011. Comparison of real-time multiplex human papillomavirus (HPV) PCR assays with INNO-LiPA HPV genotyping extra assay. J. Clin. Microbiol. 49:1907–1912. [PMC free article] [PubMed]

8. Estrade C, Menoud PA, Nardelli-Haefliger D, Sahli R. 2011. Validation of a low-cost human papillomavirus genotyping assay based on PGMY PCR and reverse blotting hybridization with reusable membranes. J. Clin. Microbiol. 49:3474–3481. [PMC free article] [PubMed]

9. Ferguson M, Wilkinson DE, Zhou T. 2009. WHO meeting on the standardization of HPV assays and the role of the WHO HPV Laboratory Network in supporting vaccine introduction held on 24-25 January 2008, Geneva, Switzerland. Vaccine 27:337–347. [PubMed]

10. Gheit T, et al. 2006. Development of a sensitive and specific assay combining multiplex PCR and DNA microarray primer extension to detect high-risk mucosal human papillomavirus types. J. Clin. Microbiol. 44:2025–2031. [PMC free article] [PubMed]

11. Gravitt PE, Peyton CL, Apple RJ, Wheeler CM. 1998. Genotyping of 27 human papillomavirus types by using L1 consensus PCR products by a single-hybridization, reverse line blot detection method. J. Clin. Microbiol. 36:3020–3027. [PMC free article] [PubMed]

12. Jacobs MV, et al. 1995. Group-specific differentiation between high- and low-risk human papillomavirus genotypes by general primer-mediated PCR and two cocktails of oligonucleotide probes. J. Clin. Microbiol. 33:901–905. [PMC free article] [PubMed]

13. Kim KH, et al. 2006. Development and evaluation of a highly sensitive human papillomavirus genotyping DNA chip. Gynecol. Oncol. 100:38–43. [PubMed]

14. Kleter B, et al. 1999. Development and clinical evaluation of a highly sensitive PCR-reverse hybridization line probe assay for detection and identification of anogenital human papillomavirus. J. Clin. Microbiol. 37:2508–2517. [PMC free article] [PubMed]

15. Klug SJ, et al. 2008. Comparison of the performance of different HPV genotyping methods for detecting genital HPV types. J. Med. Virol. 80:1264–1274. [PubMed]

16. Kocjan BJ, Poljak M, Seme K. 2010. Universal ProbeLibrary based real-time PCR assay for detection and confirmation of human papillomavirus genotype 52 infections. J. Virol. Methods 163:492–494. [PubMed]

17. Liu SS, Leung RC, Chan KK, Cheung AN, Ngan HY. 2010. Evaluation of a newly developed GenoArray human papillomavirus (HPV) genotyping assay and comparison with the Roche Linear Array HPV genotyping assay. J. Clin. Microbiol. 48:758–764. [PMC free article] [PubMed]

18. Longuet M, Beaudenon S, Orth G. 1996. Two novel genital human papillomavirus (HPV) types, HPV68 and HPV70, related to the potentially oncogenic HPV39. J. Clin. Microbiol. 34:738–744. [PMC free article] [PubMed]

19. Meijer CJ, et al. 2009. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int. J. Cancer 124:516–520. [PMC free article] [PubMed]

20. Nazarenko I, et al. 2008. A novel method of HPV genotyping using Hybrid Capture sample preparation method combined with GP5+/6+ PCR and multiplex detection on Luminex XMAP. J. Virol. Methods 154:76–81. [PubMed]

21. Nishiwaki M, et al. 2008. Genotyping of human papillomaviruses by a novel one-step typing method with multiplex PCR and clinical applications. J. Clin. Microbiol. 46:1161–1168. [PMC free article] [PubMed]

22. Resnick RM, et al. 1990. Detection and typing of human papillomavirus in archival cervical cancer specimens by DNA amplification with consensus primers. J. Natl. Cancer Inst. 82:1477–1484. [PubMed]

23. Reuter S, Delius H, Kahn T, Hofmann B, Hzur Hausen Schwarz E. 1991. Characterization of a novel human papillomavirus DNA in the cervical carcinoma cell line ME180. J. Virol. 65:5564–5568. [PMC free article] [PubMed]

24. Roberts CC, Swoyer R, Bryan JT, Taddeo FJ. 2011. Comparison of real-time multiplex human papillomavirus (HPV) PCR assays with the Linear Array HPV genotyping PCR assay and influence of DNA extraction method on HPV detection. J. Clin. Microbiol. 49:1899–1906. [PMC free article] [PubMed]

25. Schmitt M, et al. 2006. Bead-based multiplex genotyping of human papillomaviruses. J. Clin. Microbiol. 44:504–512. [PMC free article] [PubMed]

26. Schmitt M, Dondog B, Waterboer T, Pawlita M. 2008. Homogeneous amplification of genital human alpha papillomaviruses by PCR using novel broad-spectrum GP5+ and GP6+ primers. J. Clin. Microbiol. 46:1050–1059. [PMC free article] [PubMed]

27. Smith JS, et al. 2007. Human papillomavirus type distribution in invasive cervical cancer and high-grade cervical lesions: a meta-analysis update. Int. J. Cancer 121:621–632. [PubMed]

28. Snijders PJ, et al. 2006. Determination of viral load thresholds in cervical scrapings to rule out CIN 3 in HPV 16, 18, 31 and 33-positive women with normal cytology. Int. J. Cancer 119:1102–1107. [PubMed]

29. Soderlund-Strand A, Carlson J, Dillner J. 2009. Modified general primer PCR system for sensitive detection of multiple types of oncogenic human papillomavirus. J. Clin. Microbiol. 47:541–546. [PMC free article] [PubMed]

30. Sotlar K, et al. 2004. Detection and typing of human papillomavirus by e6 nested multiplex PCR. J. Clin. Microbiol. 42:3176–3184. [PMC free article] [PubMed]

31. Stambos V, et al. 2011. Evaluation of the World Health Organization global measles and rubella quality assurance program, 2001-2008. J. Infect. Dis. 204(Suppl 1):S499–S505. [PubMed]

32. Stevens MP, Garland SM, Tabrizi SN. 2008. Development and validation of a real-time PCR assay specifically detecting human papillomavirus 52 using the Roche LightCycler 480 system. J. Virol. Methods 147:290–296. [PubMed]

33. Tucker RA, Unger ER, Holloway BP, Swan DC. 2001. Real-time PCR-based fluorescent assay for quantitation of human papillomavirus types 6, 11, 16, and 18. Mol. Diagn. 6:39–47. [PubMed]

34. van den Brule AJ, et al. 2002. GP5+/6+ PCR followed by reverse line blot analysis enables rapid and high-throughput identification of human papillomavirus genotypes. J. Clin. Microbiol. 40:779–787. [PMC free article] [PubMed]

35. Walboomers JM, et al. 1999. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J. Pathol. 189:12–19. [PubMed]

36. WHO 2009. Human papillomavirus laboratory manual, 1st ed WHO, Geneva, Switzerland.

37. WHO 2006, posting date Initiative for vaccine research. Human papillomavirus. WHO, Geneva, Switzerland: http://www.who.int/biologicals/publications/trs/areas/vaccines/human_papillomavirus/HPVg%20Final%20BS%202050%20.pdf.

38. WHO 2005, posting date WHO technical workshop on the role of laboratory detection of human papillomavirus in global disease prevention and control. World Health Organization, Geneva, Switzerland: http://www.who.int/biologicals/vaccines/hpv/en/index.html.

39. Wilkinson DE, et al. 2010. Establishment of the 1(st) world health organization international standards for human papillomavirus type 16 DNA and type 18 DNA. Int. J. Cancer 126:2969–2983. [PubMed]

Articles from Journal of Clinical Microbiology are provided here courtesy of
American Society for Microbiology (ASM)