Sensitivity and Specificity of the ViroSeq Human Immunodeficiency Virus Type 1 (HIV-1) Genotyping System for Detection of HIV-1 Drug Resistance Mutations by Use of an ABI PRISM 3100 Genetic Analyzer

doi:10.1128/JCM.43.2.813-817.2005

J Clin Microbiol. 2005 February; 43(2): 813–817.

doi: 10.1128/JCM.43.2.813-817.2005

PMCID: PMC548107

Sensitivity and Specificity of the ViroSeq Human Immunodeficiency Virus Type 1 (HIV-1) Genotyping System for Detection of HIV-1 Drug Resistance Mutations by Use of an ABI PRISM 3100 Genetic Analyzer

Susan H. Eshleman,¹^* Gillian Crutcher,²^† Olga Petrauskene,³ Kevin Kunstman,⁴ Shawn P. Cunningham,¹ Christina Trevino,⁵ Cheryl Davis,⁶ John Kennedy,⁷^‡ Jeff Fairman,⁸^§ Brian Foley,⁹ and JoAnn Kop²^¶

The Johns Hopkins Medical Institutions, Baltimore, Maryland,¹ Celera Diagnostics, Alameda,² Applied Biosystems, Foster City,³ Stanford University, Palo Alto,⁵ PPGx, La Jolla,⁶ Clingenix, Sunnyvale, California,⁸ Northwestern University, Chicago, Illinois,⁴ Los Alamos National Laboratory, Los Alamos, New Mexico;,⁷ Pompano Beach, Florida⁹

^*Corresponding author. Mailing address: The Johns Hopkins Medical Institutions, Ross Building 646, 720 Rutland Ave., Baltimore, MD 21205. Phone: (410) 614-4734. Fax: (410) 614-3548. E-mail: seshlem/at/jhmi.edu.

^†Present address: Chiron, Emeryville, Calif.

^‡Deceased.

^§Present address: Juvaris BioTherapeutics, Inc., Pleasanton, Calif.

^¶Present address: Arcturus, Mountain View, Calif.

Received August 17, 2004; Revised September 23, 2004; Accepted October 23, 2004.

This article has been cited by other articles in PMC.

Abstract

The ViroSeq human immunodeficiency virus type 1 (HIV-1) genotyping system is an integrated system for identification of drug resistance mutations in HIV-1 protease and reverse transcriptase (RT). Reagents are included for sample preparation, reverse transcription, PCR amplification, and sequencing. Software is provided to assemble and edit sequence data and to generate a drug resistance report. We determined the sensitivity and specificity of the ViroSeq system for mutation detection using an ABI PRISM 3100 genetic analyzer with a set of clinical samples and recombinant viruses. Twenty clinical plasma samples (viral loads, 1,800 to 10,500 copies/ml) were characterized by cloning and sequencing individual viral variants. Twelve recombinant-virus samples (viral loads, approximately 2,000 to 5,000 copies/ml) were also prepared. Eleven recombinant-virus samples contained drug resistance mutations as 40% mixtures. One recombinant-virus sample contained an insertion at codon 69 in RT (100% mutant). Plasma and recombinant-virus samples were analyzed using the ViroSeq system. Each sample was analyzed on three consecutive days at each of three testing laboratories. The sensitivity of mutation detection was 99.65% for the clinical plasma samples and 99.7% for the recombinant-virus preparations. The specificity of mutation detection was 99.95% for the clinical samples and 100% for the recombinant-virus mixtures. The base calling accuracy of the 3100 instrument was 99.91%. Mutations in clinical plasma samples and recombinant-virus samples were detected with high sensitivity and specificity, including mutations present as mixtures. This report supports the use of the ViroSeq system for identification of drug resistance mutations in HIV-1 protease and RT genes.

The use of antiretroviral drugs has improved the health and survival of patients with human immunodeficiency virus type 1 (HIV-1) infection. Numerous reverse transcriptase (RT) inhibitors and protease inhibitors have been approved by the U.S. Food and Drug Administration (FDA) for clinical use. Unfortunately, many patients are infected with strains that are already resistant to one or more of these drugs, and many patients develop drug resistance after drug exposure. Resistance to antiretroviral drugs is usually associated with mutations in HIV-1 protease and RT genes. Genotyping and phenotyping assays have been developed for evaluation of antiretroviral drug resistance. Genotyping assays, which are based on direct detection of drug resistance mutations, are less expensive than phenotyping assays and are the most commonly used method for HIV-1 drug resistance testing. Randomized clinical trials demonstrate that use of genotypic drug resistance testing can improve the virologic response of patients to treatment regimens (1, 3).

The ViroSeq HIV-1 genotyping system (Celera Diagnostics, Alameda, Calif.) is one of two FDA-cleared systems for HIV-1 genotyping. ViroSeq is a fully integrated system that is used to identify drug resistance mutations in HIV-1 protease and the first 335 codons of HIV-1 RT. In this system, HIV-1 RNA is first isolated from 0.5 ml of plasma. This involves high-speed centrifugation followed by lysis of virus particles with a chaotropic agent to release the viral RNA and isopropanol-ethanol precipitation for RNA recovery. The RNA is reverse transcribed with Moloney murine leukemia virus RT, and the resulting DNA is amplified using a single 40-cycle PCR with AmpliTaq Gold DNA polymerase. The PCR includes the uracil N-glycosylase (UNG) contamination control system. PCR products (1.8 kb) are purified with spin columns and analyzed by agarose gel electrophoresis prior to sequencing. Premixed BigDye terminator sequencing reagents (Applied Biosystems) are used to sequence the PCR products in both directions with seven different primers. The ViroSeq software is used to assemble the sequence data into a single contiguous sequence that is used for identification of drug resistance mutations. The system reports the evidence of resistance to FDA-approved drugs based on the detection of antiretroviral drug resistance mutations. Previous studies report a high level of performance of the system for analysis of both subtype B and non-subtype B samples (2, 4, 7).

To date, the ViroSeq HIV-1 genotyping system version 2 has FDA clearance for analysis of subtype B clinical samples with viral loads of 2,000 to 750,000 copies/ml with three different automated sequencers: the ABI PRISM 377 DNA sequencer and the ABI PRISM 3100 and 3700 genetic analyzers (Applied Biosystems). The 377 instrument, which separates sequencing ladders by using polyacrylamide gels, requires manual sample loading and manual lane tracking. In contrast, the 3100 and 3700 instruments separate sequencing ladders by using capillary arrays which are automatically filled with a liquid polymer solution. Both capillary instruments load samples automatically from 96-well plates, and neither requires manual lane tracking. In this study, we determined the sensitivity and specificity of the ViroSeq HIV-1 genotyping system for the detection of HIV-1 drug resistance mutations using the ABI PRISM 3100 genetic analyzer.

MATERIALS AND METHODS

Clinical samples.

Plasma samples were collected under Institutional Review Board-approved protocols according to federal and Good Clinical Practice guidelines. Samples were collected in a blind manner and stored at −65 to −80°C in 0.5-ml aliquots. Samples with viral loads between 1,800 and 10,500 HIV-1 RNA copies/ml were further characterized by cloning the pol region by using the Invitrogen (Carlsbad, Calif.) TOPO TA cloning kit with TOP10 competent cells, according to the manufacturer's instructions. The PCR products used for cloning were generated with the ViroSeq kit without the UNG contamination control system. Forty clones from each sample were sequenced to characterize the HIV-1 variants present in each sample. Clones with stop codons or frameshift mutations were not included. Twenty plasma samples were further analyzed with the ViroSeq HIV-1 genotyping system (see below).

Recombinant-virus samples.

Twelve recombinant-virus stocks were created either by site-directed mutagenesis (9) or by cloning patient samples. The virus stocks were sequenced using the ViroSeq system. Sequencing data for each virus stock was reviewed by three independent experts to determine a consensus sequence. The recombinant-virus panel contained at least one virus stock positive for each of 43 antiretroviral drug resistance mutations (17 protease mutations and 26 RT mutations). Eleven of the recombinant-virus stocks were mixed with an HXB2 recombinant-virus stock to generate recombinant-virus samples that contained 40% mutant virus with a total viral load of approximately 5,000 copies/ml. The 12th recombinant-virus sample, which contained an insertion at RT codon 69 (T69SSS), was not mixed with the HXB2 recombinant-virus stock (100% mutant).

HIV-1 genotyping with the ViroSeq system.

To determine the sensitivity and specificity of the ViroSeq System for the detection of HIV-1 drug resistance mutations, plasma and recombinant-virus samples were analyzed using ViroSeq system version 2.0. Samples were analyzed at three laboratories: Johns Hopkins University, Northwestern University, and Celera Diagnostics. Each site had one operator and utilized a single 3100 instrument. All operators were trained in the use of the ViroSeq system and passed proficiency tests prior to the study. The sequencing reactions from each sample were analyzed three times to generate a total of nine sets of data for each sample. Genotyping was performed according to manufacturer's instructions except that sequencing reaction mixtures were resuspended in 30 μl of HiDi formamide rather than 20 μl. The larger resuspension volume permitted analysis of each sequencing reaction on three consecutive days. Sequencing analysis software (version 3.7; Applied Biosystems) was used for base calling. ViroSeq HIV-1 genotyping system software version 2.5 was used to identify drug resistance mutations and to generate drug resistance reports.

Phylogenetic analysis.

Sequences obtained in this study (nine sequences per sample) were compared to the original reference sequences for each sample. There was no evidence of sample mix-ups or cross-contamination (data not shown).

Statistical analysis.

The sensitivity and specificity of the ViroSeq system (overall and for detection of individual antiretroviral drug resistance mutations) were determined for the clinical plasma samples and recombinant-virus samples. True-positive and true-negative results for clinical plasma samples were determined based on data obtained from the analysis of clones (see below). True-positive and true-negative results for recombinant-virus samples were determined based on consensus sequences obtained for the pure virus stocks.

RESULTS

Base calling accuracy of the 3100 genetic analyzer.

This study determined the sensitivity and specificity of the ViroSeq system for the detection of HIV-1 drug resistance mutations using the ABI PRISM 3100 genetic analyzer. First, we determined the base calling accuracy of the 3100 instrument. To do this, the ABI BigDye Terminator sequencing standard (BDTSS) was run 432 times by four different operators at four different sites using nine different 3100 instruments. Because the data were not manually edited, they provided an estimate of the base calling accuracy of the 3100 instrument and Applied Biosystems sequencing analysis software version 3.7. The sequences obtained from the runs were compared to the reference sequence for the BDTSS (base positions 27 to 576); with this sequence, the Applied Biosystems software specification is >98.5% accuracy. Two data criteria were considered prior to analysis of base calling accuracy: signal intensity and peak spacing. Data from seven runs were deleted. One run was mislabeled, five runs had signal intensities of <10, and one run had a peak spacing which did not meet specifications (range, 11 to 16). In this study, the overall all-base accuracy for bases 27 to 576 was 99.91% (233,539 of 233,750 bases, 211 bases miscalled) with a standard deviation of 0.33% for the 425 BDTSS sequences analyzed. Of the 425 analyzed sequences, 335 (78.9%) had an accuracy of 100%; 88 (20.7%) had an accuracy between 99 and 100%; 1 (0.2%) had an accuracy of 98.54%; and 1 (0.2%) had an accuracy of less than 98.5% (95.08%, with 27 bases miscalled).

Analysis of a clinical plasma sample panel.

To determine the sensitivity and specificity of the ViroSeq system, we first generated a panel of clinical plasma samples containing antiretroviral drug resistance mutations in HIV-1 protease and RT. The viral loads of the samples ranged from 1,800 to 10,500 HIV-1 RNA copies/ml. Prior to analysis with the ViroSeq system, the plasma samples were characterized by cloning and sequencing individual viral variants. This process served to confirm the presence of individual mutations in the samples and to assess the prevalence of each mutation in the viral population. Approximately 40 clones from each sample were sequenced (range, 30 to 50 clones) (Tables (Tables11 and and2).2). Mutations detected in at least 10 clones were designated true-positive results. This threshold was established to meet a goal, namely, that mutations present in approximately 40% of the viral population would be detected in the ViroSeq system with 95% confidence. Some mutations that were present at lower levels (in one to nine clones per sample) were also detected by the ViroSeq system. Those results were also considered true-positive results, since the cloning analysis documented that the mutations were present in the sample. The ViroSeq software version 2.5 algorithm for the interpretation of HIV-1 genotyping considers 75 mutations (35 in protease and 40 in RT). Forty of those mutations (21 in protease and 19 in RT) were present in 10 or more clones in at least one of the plasma samples. Ten other mutations were present in one to nine clones from at least one sample, and 23 mutations were not represented in the sample set (Tables (Tables11 and and2).2). The RT mutations G333D and G333E were not analyzed.

TABLE 1.

Characterization of protease mutations in clinical plasma samples^a

TABLE 2.

Characterization of RT mutations in clinical plasma samples^a

Plasma samples were analyzed in triplicate using the ViroSeq system in each of three laboratories to generate nine sets of genotyping data for each of the 20 samples. Data from the three testing laboratories were combined to determine the sensitivity and specificity of detection of each protease and RT mutation. For example, the L10I mutation was represented in five samples (i.e., five samples with ≥10 clones positive for the mutation). With nine runs, there were 45 positive and 135 negative data points available for evaluation.

In this study, the overall sensitivity of the ViroSeq system for mutation detection in the plasma sample panel was 99.65% for 40 of the 75 mutations in the ViroSeq version 2.5 software algorithm. All but one of the 40 mutations represented in the panel (i.e., with ≥10 clones positive for the mutation in one or more of the samples) was detected in all nine runs (sensitivity, 100%). The one exception was the M41L mutation in RT. This mutation was not detected in sample 12 in two laboratories (missed in four of nine runs). Sample 12 had 11 of 38 clones with the M41L mutation. The same mutation was correctly identified in five other samples (1, 13, 14, 17, and 18) in all nine runs. Therefore, the sensitivity of detection of M41L in the plasma sample panel was 92.6% in this study.

The specificity of the ViroSeq system for mutation detection was calculated for 73 of the 75 mutations that are evaluated in the ViroSeq software version 2.5 algorithm for genotype interpretation. Two mutations in RT (G333D and G333E) were not considered. The overall specificity of the ViroSeq system for the detection of mutations in the plasma sample panel was 99.95% in this study. The specificity of detection for all but one of the mutations was 100%. The exception was the L63P mutation in protease. Two sites identified L63P in sample 6 in all three runs. However, the mutation was not detected among the 36 clones analyzed from that sample. Therefore, the specificity for detection of the L63P mutation was 93.9% in this study.

Analysis of recombinant-virus samples.

Twelve recombinant-virus samples were analyzed with the ViroSeq system in triplicate in each of the three laboratories to generate nine sets of genotyping data for each sample. Eleven samples were used to determine the sensitivity and specificity of the ViroSeq system for the detection of mutations in viruses that represented 40% of the viral population (mixtures) at a viral load of 2,000 to 5,000 copies/ml (see Materials and Methods). The twelfth sample was used to determine the sensitivity and specificity of ViroSeq for the detection of an insertion at codon 69 in RT, present at 100% of the viral population (no mixture; see Materials and Methods).

The 12 recombinant-virus samples included 43 antiretroviral drug resistance mutations (17 in protease and 26 in RT). In this study, the overall sensitivity of the ViroSeq system for detection of those 43 mutations in the recombinant-virus samples was 99.7%. Forty-two of 43 mutations were detected with 100% sensitivity. The one exception was the A62V mutation in RT. This mutation was missed in three of the nine runs of sample 3 in a single laboratory. For two other samples (10 and 11), the A62V mutation was detected in all runs. The sensitivity of detection of the A62V mutation as a 40% mixture was 88.9%.

The specificity calculation for the ViroSeq system for mutation detection utilized 73 of the 75 mutations in the ViroSeq software algorithm (same as those considered in the analysis of the clinical plasma samples [see above]). This included 18 protease mutations and 12 RT mutations that were not represented in the recombinant-virus samples. In this study, the overall specificity for the recombinant-virus sample panel was 100%.

DISCUSSION

ViroSeq HIV genotyping system version 2 is a fully integrated system for analysis of HIV-1 drug resistance mutations. Reagents, protocols, and software are provided for all steps of analysis, and the assay is performed on open-system equipment that can be used for other applications. In addition to providing an interpretive drug resistance report, the software provides an uninterrupted sequence encoding protease and the first 335 amino acids of RT in the universal FASTA format, simplifying phylogenetic analysis for quality control and other studies. ViroSeq system version 2.5 was cleared by the FDA for diagnostic use with the ABI PRISM 377 DNA sequencer in 2002. The data that supported the FDA clearance with the 377 instrument included 100 clinical samples. Fifty samples had viral loads from 1,800 to 10,500 copies/ml. Those 50 low-viral-load samples were characterized by cloning and sequencing individual viral variants prior to analysis with the ViroSeq system. Twenty of the low-viral-load samples were selected for the analysis in this study. Data reported in this study supported the FDA clearance of the ViroSeq system for analysis using the 3100 instrument in 2003. The 3100 instrument is a capillary-based instrument that loads samples automatically from 96-well plates. Lane tracking is not required. The 3100 instrument, which uses a 16-capillary array, can be loaded with two 96-well plates and can process 16 reactions in 2.5 h.

In this study, the all-base accuracy of the 3100 instrument without manual intervention (editing) was 99.9% with the Applied Biosystems BDTSS. For bases 27 to 576, 424 of the 425 sequences met the Applied Biosystems specification of >98.5% accuracy. This high accuracy of base calling, along with the double coverage of ViroSeq sequences and the ViroSeq software's ability to perform manual edits, resulted in highly accurate sample consensus sequences.

In this study, the overall sensitivity and specificity of ViroSeq for detection of drug resistance mutations were 99.65 and 99.95%, respectively, for clinical samples, and were 99.7 and 100%, respectively, for recombinant samples containing mutations as 40% mixtures with viral loads of approximately 5,000 copies/ml. Similar results were obtained at three independent testing sites. These performance characteristics are equivalent to those obtained previously using the 377 instrument (ViroSeq HIV-1 Genotyping System Operator's Manual, 2003, CDx, part number 4341639). The sensitivity for mutation detection was determined for 73 of the 75 mutations included in the ViroSeq resistance interpretation algorithm. The sensitivity was 100% for all but two of the mutations (M41L and A62V in RT), and the specificity was 100% for all but one mutation (L63P in protease). For these mutations, the triplicate sequences were consistent, suggesting that detection of the mutation was dependent on a presequencing event. Since these samples were low-viral-load samples (2,000 to 10,500 copies/ml), it is possible that (i) cDNA from the RT reaction was biased due to a less-than-40% mixture or (ii) the sequence context of the mutation affected dideoxynucleoside triphosphate incorporation, such that some mutations were not proportional to the population mixture. This study suggests that the following criteria are important for the generation of high-quality HIV-1 genotypes: (i) the RT-PCR yield should be at least 20 ng per 5 μl; (ii) total sequencing base signals should be ≥400; (iii) peak spacing should be between 11 and 16; and (iv) the ViroSeq positive control (8E5) should show no mixtures.

When genotyping assays are performed, it is important to minimize contamination of reaction mixtures with previously amplified products. Sample cross-contamination is minimized in the ViroSeq system by using a single nonnested PCR for amplification, and by using a dUTP/UNG contamination control system. There was no evidence of sample cross-contamination in this study or in previous reports (2, 4).

This study supports use of the ViroSeq system and ABI PRISM 3100 genetic analyzer for identification of HIV-1 drug resistance mutations. Testing is performed using only 0.5 ml of plasma. Research methods have also been developed to use PCR products generated with ViroSeq for analysis of individual HIV-1 variants and genetic linkage of drug resistance mutations by cloning (5), for analysis of the HIV-1 gag region (6), and for detection of minority HIV-1 variants with drug resistance mutations (8). The system is currently available for diagnostic use in both the United States and Europe. The system is cleared by the FDA for analysis of subtype B HIV-1 only. However, excellent performance of the system has also been documented in research studies for analysis of diverse HIV-1 strains (4). An updated version of the ViroSeq HIV-1 genotyping system (version 2.6) was recently cleared by the FDA. Version 2.6 includes an updated algorithm for interpretation of drug resistance to all currently licensed protease and RT inhibitors. The results described in this report should be relevant to version 2.6 as well as version 2.5, since these versions are identical in all aspects except for the interpretive report.

Acknowledgments

S.H.E. was supported in part by (i) the HIV Prevention Trials Network (HPTN) sponsored by the NIAID, National Institutes of Child Health and Human Development (NICH/HD), National Institute on Drug Abuse, National Institute of Mental Health, and Office of AIDS Research of the NIH, DHHS (grants U01-AI-46745 and U01-AI-48054), and (ii) the Adult AIDS Clinical Trials Groups (Division of AIDS, NIAID, NIH; grant U01-AI38858).

REFERENCES

1. Baxter, J. D., D. L. Mayers, D. N. Wentworth, J. D. Neaton, M. L. Hoover, M. A. Winters, S. B. Mannheimer, M. A. Thompson, D. I. Abrams, B. J. Brizz, J. P. Ioannidis, T. C. Merigan, et al. 2000. A randomized study of antiretroviral management based on plasma genotypic antiretroviral resistance testing in patients failing therapy. AIDS 14:F83-F93. [PubMed]

2. Cunningham, S., B. Ank, D. Lewis, W. Lu, M. Wantman, J. Dileanis, J. B. Jackson, P. Palumbo, P. Krogstad, and S. H. Eshleman. 2001. Performance of the Applied Biosystems ViroSeq human immunodeficiency virus type 1 (HIV-1) genotyping system for sequence-based analysis of HIV-1 in pediatric plasma samples. J. Clin. Microbiol. 39:1254-1257. [PMC free article] [PubMed]

3. Durant, J., P. Clevenbergh, P. Halfon, P. Delgiudice, S. Porsin, P. Simonet, A. Montagne, C. A. B. Boucher, J. M. Schapiro, and P. Dellamonica. 1999. Drug-resistance genotyping in HIV-1 therapy: the VIRADAPT randomised controlled trial. Lancet 353:2195-2199. [PubMed]

4. Eshleman, S. H., J. Hackett, Jr., P. Swanson, S. P. Cunningham, B. Drews, C. Brennan, S. G. Devare, L. Zekeng, L. Kaptué, and N. Marlowe. 2004. Performance of the Celera Diagnostics ViroSeq HIV-1 genotyping system for sequence-based analysis of diverse human immunodeficiency virus type 1 strains. J. Clin. Microbiol. 42:2711-2717. [PMC free article] [PubMed]

5. Eshleman, S. H., D. Jones, T. Flys, O. Petrauskene, and J. B. Jackson. 2003. Analysis of HIV-1 variants by cloning DNA generated with the ViroSeq HIV-1 genotyping system. BioTechniques 35:614-622. [PubMed]

6. Marlowe, N., T. Flys, J. Hackett, M. Schumaker, J. B. Jackson, and S. H. Eshleman. 2004. Analysis of insertions and deletions in the gag p6 region of diverse HIV-1 strains. AIDS Res. Hum. Retrovir. 20:1119-1125. [PubMed]

7. Mracna, M., G. Becker-Pergola, J. Dileanis, L. A. Guay, S. Cunningham, J. B. Jackson, and S. H. Eshleman. 2001. Performance of the Applied Biosystems ViroSeq HIV-1 genotyping system for sequence-based analysis of non-subtype B human immunodeficiency virus type 1 from Uganda. J. Clin. Microbiol. 39:4323-4327. [PMC free article] [PubMed]

8. Shi, C., S. H. Eshleman, D. Jones, N. Fukushima, L. Hua, A. R. Parker, C. J. Yeo, R. H. Hruban, M. G. Goggins, and J. R. Eshleman. 2004.. LigAmp: sensitive detection of single nucleotide differences. Nat. Methods 1:141-147. [PubMed]

9. Short, M., J. Chen, A. Devitt, M. Furtado, C. Brown, and the HIV Genotyping Team. 2002. Construction of molecular infectious clones (MICs) and viral panels with defined mixture composition to specify guidelines for editing mixed base positions during population sequencing of HIV-1. Presented at the XIV International AIDS Conference, Barcelona, Spain.

Articles from Journal of Clinical Microbiology are provided here courtesy of
American Society for Microbiology (ASM)