This is the first large-scale study evaluating the quality of sequence analysis approaches for the identification of drug resistance mutations in the HIV-1 RT gene. Recent developments and improvements in genotyping technologies (9
), a rapidly evolving understanding of the effects of resistance mutations on viral drug susceptibility, and the fact that biological drug susceptibility assays are costly, difficult to standardize, and extremely time-consuming (2
) have led to an increasing use of genotypic drug resistance methods for HIV. At present these methods are mainly applied by research laboratories but are beginning to be implemented diagnostically. Most importantly, genotyping to determine the resistance profile in patients virologically showing treatment failure has recently been shown to be beneficial in selecting a salvage regimen (1
), and as such, sequencing results become of increasing importance in the clinical care of HIV-1-infected patients.
This first study evaluating the quality of HIV drug resistance genotyping protocols was based on the use of recombinant RT genes derived from a reference virus, into which mutations had been introduced via site-directed mutagenesis. This approach enabled a direct comparison of the reliability of DNA sequencing protocols in the absence of factors such as variation introduced by viral quasispecies, the nucleic acid extraction procedure, or the cDNA synthesis reactions. Participants used their standard sequencing hardware, procedures, and protocols to analyze the panel samples. Most of the participants used an ABI sequencer (17 of 23 labs); the remaining six labs used the additional available sequencing technologies. This distribution of technologies therefore did not allow the comparison of the results for each hardware technology. Moreover, the qualitative interpretation of the results demonstrated highly variable scores even between laboratories using the same sequencing technology, indicating that interlaboratory differences are extensive and may affect the results more than the differences between the technologies. The striking differences in the M values show that large differences exist in the quality of DNA sequencing results of laboratories dedicated to performing these types of analyses routinely on a research or even diagnostic basis.
The qualitative interpretation of the laboratory results for each of the panel samples demonstrated that none of the panel samples, including those containing only wild-type or mutant genotypes, was scored correctly by all of the laboratories. Incorrect codon calls, i.e., the detection of a codon sequence that did not reflect the wild-type or the mutant sequence, were reported for several of the panel samples, in particular for codons 41 and 184, independent of the mutant concentration. However, all these calls originated from three (13%) of the participants. No incorrect codons at the mutation sites were reported by any of the other participants.
In addition, the results for codons 41 and 215 demonstrated that some laboratories missed the presence of a resistance mutation, even in samples with a homogeneous genetic makeup, or in the opposite situation, the presence of a mutation was reported in samples containing a complete wild-type input. It is of note that the participant detecting the codon 41 mutation, when it was present at the level of 5%, also reported the presence of this mutation in the purely wild-type specimen. This suggests that the reported detection of the 5% minority species might result from a nonspecific sequencing reaction or sample contamination and does not necessarily reflect a highly sensitive and specific procedure.
Upon entry of the data into the database all participants reviewed the blinded results for data entry errors. It cannot be excluded that some errors may have been introduced somewhere in the entire multistep laboratory process or at the steps of data analysis, data reporting, and data entry into the central database. Since, except for data entry into a central database, all these procedures are part of the standard sample manipulation process in HIV-1 genotyping, the results may be a good reflection of the actual overall quality of these procedures in daily practice.
The analysis of the results did not include an evaluation of the quality of the sequence reaction, e.g., signal strength or peak heights, and its effect on the sequence interpretation. Since the decision to use the sequence results for interpretation was in the hands of the participants, without any predefined quality criteria, differences in the quality of the sequence reactions itself may partially explain the observed interlaboratory differences.
The results of this study indicate that before applying DNA sequencing diagnostically for the detection of drug resistance mutations, considerable improvements need to be made in the quality of the results and procedures. In order to achieve this and monitor the quality of these procedures on a continuous basis, the installation of proper quality control programs and the standardization of protocols are essential. A significant improvement in the overall quality may also come from the use of dedicated kits and procedures to perform HIV-1 drug resistance genotyping, as recently introduced by ABI and Visible Genetics (Toronto, Canada).
The sensitivity of detecting a mutant genotype when this was present at a relative concentration of 25% was comparable for the mutations at codons 41 and 215 and for the 184-valine mutation. Interestingly, a much lower frequency was observed for the 184-isoleucine mutation. This might be due to differences in the sensitivities of the sequencing procedures to detect each of the nucleotide changes. The 184-isoleucine mutation is the result of a G→A mutation, whereas for all three other codons at least one of the base changes is due to a mutation in which the wild-type A nucleotide is replaced (codon 41, A→G; codon 215, A→T; and C→A, codon 184Val A→G). The relatively low sensitivity for codon 184-isoleucine was specifically observed in a combination with both the wild type and valine variant, indicating that the high level of variation in the codon might have affected the interpretation of the analyzed nucleotide sequence. Another explanation might be that the codon 184-isoleucine mutation was unexpected by the participants, since the mutation is rather infrequent in clinical isolates from extensively lamivudine-treated patients (8
As mentioned before, the inputs for codons 41 and 215 were identical, as the mutations were coupled on the source plasmid. The concordance of results for codons 41 and 215, based on the qualitative results, was high, though it was complete in none of the samples. The maximum variation, i.e., a 50% wild-type and mutant mixture, resulted in only 79% concordance, again suggesting mutation-specific differences in mutation detection sensitivities. Apart from possible explanations mentioned before, this difference might also be due to other factors such as the distance of the mutation from the sequencing primer, resulting in divergent signal strengths at the location of the mutations.
The quantitative determination of the mutations demonstrated that laboratories capable of detecting a mutation qualitatively generally estimated its relative concentration with 25% accuracy from the input concentration. This indicates that laboratories performing good quality sequencing should be able to differentiate relative mutant concentrations in strata of 25%.
This study demonstrates that extensive differences exist in the quality of DNA sequence analysis for the identification of HIV-1 drug resistance mutations. Although the capacity to determine mutations was analyzed only for the HIV-1 RT gene, there is no reason to assume that the quality of results would be different for the protease gene or other target genes and organisms. In summary, this first multicenter evaluation of DNA sequence analysis procedures for HIV-1 drug resistance demonstrates large differences in the overall quality of the results. Many of the laboratories generated moderate to good results, while none of the labs were perfect. A small number of laboratories demonstrated poor performance. Therefore, the clinical application of DNA sequencing results should be considered with care, and clinicians should be clearly educated about the current limitations of sequencing technology for HIV-1 drug resistance genotyping.
As an important step towards the improvement of the quality of the results, the development of quality control programs for genotyping is essential.