|Home | About | Journals | Submit | Contact Us | Français|
The purpose of this study was to evaluate the performance of laboratories for the detection and quantification of human herpesvirus 6 (HHV-6) by an external quality assessment (EQA) evaluation. The HHV-6 EQA panel consisted of eight samples containing various concentrations of HHV-6 type A (strain GS) or type B (strain Z29), two samples containing other herpesviruses (i.e., human cytomegalovirus [HCMV] and Epstein-Barr virus [EBV]), and two HHV-6-negative samples. Panel samples were prepared in human plasma, heat inactivated, and lyophilized. Panel distribution, data management, and analysis were coordinated by Quality Control for Molecular Diagnostics (QCMD), Glasgow, United Kingdom. Fifty-one laboratories participated and submitted 57 data sets. Eleven (19.3%) data sets were generated using conventional in-house assays, 11 (19.3%) data sets using commercial real-time PCR assays, and 35 (61.4%) data sets using in-house real-time PCR assays. The presence of HHV-6 DNA at viral loads exceeding 6,000 copies/ml was detected by all participants, and over 80% of the participants still reported correct qualitative results for the sample containing just over 200 copies/ml. The false-positivity rate was 1.8% for both the negative samples and the samples containing HCMV or EBV DNA. The majority (23/33; 69.7%) of quantitative data sets were generated using in-house real-time PCR assays. The standard deviations of the geometric means of the samples ranged from 0.5 to 0.7 log10. The results of this first international EQA demonstrate encouraging analytical sensitivity for the detection of HHV-6-DNA in human plasma, although we observed extensive interlaboratory variation of quantitative HHV-6 DNA results. Standardization needs to be improved to allow further elucidation of the clinical significance of HHV-6 loads.
Human herpes virus type 6 (HHV-6) is a member of the Betaherpesvirus subfamily (genus Roseolovirus). Two distinct variants have been described: HHV-6 type A (HHV-6A) and HHV-6B. HHV-6 type B infection is recognized as the cause of a febrile disease and exanthem subitum in early childhood. Over 90% of the population worldwide is infected within the first 18 months of life (5, 12). After primary infection, HHV-6 establishes life-long persistence in the host and is detectable in multiple tissues, similar to other herpesviruses, such as human cytomegalovirus (HCMV) (6). HHV-6 infection rarely causes severe disease in healthy children, but viral reactivation in immunocompromised patients is associated with severe morbidity (encephalitis and acute graft-versus-host disease) and increased mortality (11). The majority of HHV-6 infections are caused by HHV-6 type B (2, 9). Although HHV-6 type A is rarely detected, this variant is suggested to act as a more neurotropic pathogen in the context of severe central nervous system infections (4).
Because of their speed, sensitivity, and specificity, molecular diagnostic assays are increasingly used for the detection of HHV-6 DNA to diagnose viral disease. Various commercial and in-house assays are currently available. Due to the lack of any available well-characterized reference reagents, most of these assays lack standardization, both in performance as well as in viral load calculation.
This lack of standardization causes difficulties in comparing results between laboratories and, as a consequence, complicates the clinical interpretation of laboratory results for HHV-6. In an attempt to compare the results for HHV-6 DNA detection and quantitation between laboratories, a first international external quality assessment (EQA) study was organized.
To assess the proficiency of laboratories in the detection and quantification of HHV-6 by nucleic acid amplification, a blinded panel of 12 samples containing various concentrations of HHV-6 types A and B as well as negative controls and specificity controls was sent to laboratories worldwide. The laboratories were instructed to analyze the panel according to their local routine procedure for HHV-6 PCR detection and quantitation.
Samples were prepared at the Department of Virology, University Medical Center Utrecht, Utrecht, Netherlands, by serial dilution of HHV-6 type A strain GS and HHV-6B strain Z29 (Advanced Biotechnologies Inc., MD). The concentration of virus particles in the viral stocks of the source viruses had been previously determined by electron microscopy (Advanced Biotechnologies Inc.). The virus stocks were heat inactivated at 56°C for 30 min and kept at −80°C before they were used for the preparation of the EQA panel. Preparation of serial bulk dilutions of the viral stocks was performed in HHV-6-, Epstein-Barr virus (EBV)-, and HCMV-negative human plasma in EDTA. The target concentrations of the panel samples ranged from 200 to 25,000 (log10 2.3 to 4.4) copies/ml. The bulk dilutions were sent on dry ice to Quality Control for Molecular Diagnostics (QCMD) in Glasgow, United Kingdom, where 1-ml aliquots were prepared by QCMD and subsequently lyophilized prior to distribution at ambient temperature.
The final panel consisted of eight HHV-6-containing human plasma samples, two plasma samples from healthy individuals, and two plasma samples containing HCMV or EBV at a concentration of log10 4.4 (25,000) copies/ml to assess the specificities of the assays.
The panel samples were coded, as indicated in Table Table1.1. The participants were instructed to analyze the samples according to their standard laboratory protocols. The results were reported to the QCMD neutral office online via a data submission form on the QCMD website within 6 weeks of the receipt of the panel. To ensure confidentiality, all participating laboratories received a code number for data reporting and further communication.
The QCMD neutral office confidentially handled the coding of the panel, collected the results and questionnaires, and analyzed the results, as described before (8).
The data were analyzed according to qualitative and quantitative criteria. In addition, the results were analyzed and compared by type of technology, i.e., in-house-developed versus commercial assays and conventional versus real-time PCRs.
A quantitative geometric mean value was calculated for each panel sample in two ways, namely, as the consensus concentration (the geometric mean of the participants' results, once outliers had been removed) and as the technology consensus concentration (the geometric mean of the participants' results per technology group once, outliers had been removed). For each sample in a data set of quantitative results, scores were awarded on the basis of the distance from the calculated geometric mean value for each panel sample. Zero points were awarded if the quantitative value returned was within 1 standard deviation (SD) from the geometric mean. One point was awarded if the quantitative value was between 1 and 2 SDs, 2 points if the value was within 2 and 3 SDs, and 3 points for quantitative values more than 3 SDs from the geometric mean. For each data set, a cumulative performance score was calculated on the basis of the sum of the performance scores of the individual samples in that data set. Thus, zero was the best score that could be achieved for the panel as a whole; i.e., all results within 1 SD from the geometric means were calculated for the samples (7).
Fifty-one laboratories from 20 countries (the number of participating laboratories per country: Australia, 2; Austria, 2; Belgium,1; Canada, 1; Czech Republic, 2; Finland, 1; France, 4; Germany, 3; Israel, 2; Italy, 2; Lebanon, 1; Netherlands, 6; New Zealand, 1; Norway, 1; Portugal, 3; South Africa, 2; Sweden, 4; Switzerland, 4; United Kingdom, 5; United States, 4) participated in this first international HHV-6 EQA program in 2008. In total, 57 completed data sets were received from 50 participants, 2 laboratories reported 2 data sets, and 1 laboratory reported 3 data sets. One participant withdrew, reporting that the assay was still under development. Eleven (19.3%) data sets were generated using conventional in-house assays, 11 (19.3%) data sets using commercial real-time PCR assays, and 35 (61.4%) data sets using in-house real-time PCR assays (Table (Table1).1). A range of nucleic acid extraction methods have been used. No significant differences in correct results between these methods were found (Fig. (Fig.1).1). In this study, the heterogeneity of the nucleic acid extraction methods used did not allow further in-depth analysis of the results.
Overall, the data sets obtained with real-time commercial technologies (n = 11) reported more correct results for the positive samples than the data sets obtained with in-house real-time and conventional in-house technologies (100% versus 96.1%), indicating that they have a higher sensitivity than some in-house-developed assays (Table (Table11).
The consensus geometric mean viral load for the samples ranged from approximately 25,800 to 240 (log10 4.411 to 2.383) copies/ml, closely matching the target concentrations that were defined prior to preparation of the panel. All participants were able to detect HHV-6 DNA in the samples containing 6,500 (log10 3.814) copies/ml or more. A viral load of 240 (log10 2.384) copies/ml was still reported positive by 86% of the participants. For the commercial real-time PCR-based assays, 11/11 (100%) data sets correctly detected the presence of HHV-6 DNA in the positive sample with the smallest amount of HHV-6 DNA (242 copies/ml). No differences in results were found for HHV-6A and HHV-6B DNA. The performance rates declined with lower HHV-6 concentrations in the samples. The overall proportion of false-negative results was 15/456 (3.5%). All these false-negative results were reported by two laboratories.
Two false-positive results were reported for the negative samples (2/114, 1.8%), one from an in-house conventional PCR and the other from an in-house real-time PCR. The specificities of the assays for the detection of HHV-6, as shown by the results for samples with other herpesviruses (HCMV and EBV), was high. A false-positive result was reported only twice, and therefore, the percentage of false-positive results for the HCMV- and EBV-positive samples was the same as that observed for the true-negative samples (1.8%).
In 44/57 (77%) data sets, correct qualitative results were reported for each of the panel samples.
Thirty-three (58%) data sets based on quantitative results were submitted. The geometric means and the ranges of the reported viral load results are shown in Table Table2.2. The majority of the data sets with quantitative results were obtained with in-house real-time PCR assays (24/33; 73%). Nine data sets were obtained with a commercial assay/kit. The mean viral load results obtained with commercial methods were similar to those obtained with the in-house-developed technologies for HHV-6 type B but were approximately 3-fold lower for HHV-6 type A (Table (Table2).2). Furthermore, the ranges of the viral load results were wider for the HHV-6 type A results obtained with commercial assays than for the results obtained with in-house assays (Table (Table2).2). The ranges for the HHV-6 type A results generated with commercial assays started at concentrations lower than those for in-house assays. This difference was not observed for HHV-6 type B results.
The variation in viral loads per sample was large, with the standard deviations of the log10 number of copies/ml ranging from 0.5 to 0.7. However, in general, the laboratories were correctly able to rank consecutive samples with differences in HHV-6 loads (Fig. (Fig.2).2). In 6/33 (18%) data sets, the reported viral load was within 1 SD from the geometric mean for each panel sample. In an additional five data sets, the results were within 1 SD for all except one sample, the latter being <2 SDs from the geometric mean.
For 8/33 (24%) data sets, quantitative results were not reported for one or more samples, and therefore, overall performance scores could not be calculated (Fig. (Fig.33).
The program described here is the first international EQA program to evaluate the detection and quantification of HHV-6 DNA. Fifty-one laboratories participated, underlining the need for an EQA program in this area. The majority of the HHV-6 DNA detection assays used by the participants are sensitive and specific. All participants obtained excellent results for qualitative performance, but the quantitative results clearly need improvement.
The use of well-validated and standardized assays for the detection of HHV-6 DNA in clinical samples is important to evaluate the clinical significance of HHV-6. However, an international HHV-6 standard is not yet available.
More than 80% of the participants recorded correct qualitative results for the sample containing the lowest viral concentration (approximately 200 copies/ml), indicating that the overall sensitivity of the nucleic acid amplification methods for the detection of HHV-6 is at least comparable to the sensitivity level generally accepted for other herpesviruses, such as HCMV and EBV, or even better. In comparison, the sensitivities for samples with similar viral loads in QCMD's 2008 EQA program for HCMV and EBV were only 61.3% and 77.8%, respectively. Overall, the commercial assays appeared to be slightly more sensitive than some in-house assays (conventional or real-time PCR assays).
In this study, 81% data sets were obtained with in-house assays, the majority of which (65%) were based on real-time PCR. In-house assays are usually based on different assay setups and often vary in virtually all aspects of the procedure, including nucleic acid extraction, amplification, primers, probes, etc. Remarkably, the concentrations reported for the HHV-6 type A samples were, on average, higher among participants using real-time in-house-developed assays than commercial real-time PCR assays, although the difference was not significant. This difference was not observed for the HHV-6 type B samples (Table (Table2).2). Flamand et al. previously analyzed nine molecular HHV-6 detection assays systematically by using HHV-6-spiked serum samples with between 0 and 100,000 HHV-6 genomes copies per sample (3). In their study, three TaqMan-based PCR assays showed results closest to the expected values (56,406 to 172,000 for the three TaqMan-based PCR assays versus 16,707 to 245,250 expected genome copies) (3). In our HHV-6 EQA program, we evaluated more laboratories and different types of assays, but our results were not in agreement with the TaqMan assay-based results of Flamand et al. (3). Our EQA program was primarily designed to assess a laboratory's ability to detect HHV-6 DNA correctly rather than test the performance of in-house versus commercial assays. Such a comparison is possible only if each method is used in sufficient numbers and would probably require a different study design. In this study, the number of quantitative commercial assays (9/33, 27%) did not allow such a comparison; a future distribution with higher numbers of participants may allow such a comparison.
Additionally, different PCRs which amplify different HHV-6 gene regions were used in our EQA. The differences in the quantitative and qualitative results might be explained by these differences in assay setup. To date, calibration of assays is not easily achieved, due to a lack of standardized reference reagents. Further standardization is needed to allow comparison of quantitative results between different laboratories and to allow elucidation of the clinical role of the HHV-6 load.
The increasing demand for rapid and reliable methods for the diagnosis of HHV-6 disease has led to the widespread introduction of molecular diagnostic procedures into the clinical virological laboratory. Different studies have shown an association between increased HHV-6 load and clinical disease (e.g., encephalitis) (10) or complications after hematopoietic stem cell transplantation (1, 11). With standardized reference materials, the different molecular HHV-6 assays might be evaluated and the sensitivity and specificity may further improve. Finally, this may lead to a comparison of quantitative results between different laboratories and may allow elucidation of the clinical role of the HHV-6 load.
We thank all participants of this first HHV-6 EQA program coordinated by QCMD.
We all declare that we have no conflict of financial interest.
Published ahead of print on 10 February 2010.