|Home | About | Journals | Submit | Contact Us | Français|
Background.Viral load (VL) quantification is considered essential for determining antiretroviral treatment (ART) success in resource-rich countries. However, it is not widely available in resource-limited settings where the burden of human immunodeficiency virus infection is greatest. In the absence of VL monitoring, switches to second-line ART are based on World Health Organization (WHO) clinical or immunologic failure criteria.
Methods.We assessed the performance of CD4 cell criteria to predict virologic outcomes in a large ART program in Nigeria. Laboratory monitoring consists of CD4 cell count and VL at baseline, then every 6 months. Failure was defined as 2 consecutive VLs >1000 copies/mL after at least 6 months of ART. Virologic outcomes were compared with the 3 WHO-defined immunologic failure criteria.
Results.A total of 9690 patients were included in the analysis (median follow-up, 33.2 months). A total of 1225 patients experienced failure by both immunologic and virologic criteria, 872 by virologic criteria only, and 1897 by immunologic criteria only. The sensitivity of CD4 cell criteria to detect viral failure was 58%, specificity was 75%, and the positive-predictive value was 39%. For patients with both virologic and immunologic failure, VL criteria identified failure significantly earlier than CD4 cell criteria (median, 10.4 vs 15.6 months; P < .0001).
Conclusions.Because of the low sensitivity of immunologic criteria, a substantial number of failures are missed, potentially resulting in accumulation of resistance mutations. In addition, specificity and predictive values are low, which may result in large numbers of unnecessary ART switches. Monitoring solely by immunologic criteria may result in increased costs because of excess switches to more expensive ART and development of drug-resistant virus.
International and national programs have supported the rapid expansion of antiretroviral therapy (ART) access in countries with the highest burden of human immunodeficiency virus (HIV) infection. Despite significant progress, with >5.2 million individuals receiving ART, the World Health Organization (WHO) estimates that only one-third of persons in need of ART have been reached . As ART programs continue to expand, achieving balance between cost constraints and quality of care remains at the forefront of sustainability discussions.
In resource-rich countries, plasma viral load (VL) monitoring is considered to be the gold standard for assessing treatment success [2, 3]. Historically, use was limited even in developed nations until sufficient evidence confirmed that VL monitoring improved patient outcomes while also proving to be cost-effective [4–6].
The optimal use of laboratory monitoring for patients receiving ART in resource-limited settings (RLS) remains controversial [7–11]. However, evidence suggests that immunologic and clinical criteria are inadequate to predict virologic failure [12–15]. Limiting aspects of previous studies determining the usefulness of immunologic and clinical parameters as predictors of virologic failure include short follow-up times and overall low virologic failure rates resulting in predictive values that may not be generalizable to actual treatment programs.
As the debate continues regarding how to best use resources for treatment monitoring, given the ongoing need to expand access, this study was conducted to evaluate the ability of the WHO immunologic criteria to predict failure in a large treatment cohort in Nigeria with a longer duration of treatment follow-up than in previous studies.
The Harvard President’s Emergency Plan for AIDS Relief (PEPFAR)/AIDS Prevention Initiative in Nigeria (APIN) program has provided HIV care services to >100 000 individuals at 33 clinical sites in Nigeria since 2004. This retrospective cohort study evaluates treatment data from patients enrolled at 4 tertiary hospitals: Jos University Teaching Hospital, the National Institute of Medical Research, 68 Military Hospital, and University of Maiduguri Teaching Hospital. ART eligibility was based on the Nigerian National Adult ART Guidelines , with ART recommended for all individuals with CD4 cell counts ≤200 cells/mm3 and for those individuals with CD4 cell counts ≤350 cells/mm3 and clinical stage 3 or 4 conditions. Recommended first-line ART included a nonnucleoside reverse-transcriptase inhibitor (NNRTI) plus 2 nucleoside reverse-transcriptase inhibitors.
The study cohort consisted of ART-naive individuals >15 years of age for whom at least 12 months of treatment follow-up was available. In addition, a minimum number of laboratory evaluations were necessary to assess failure criteria, including CD4 cell count at baseline and after at least 12 months of ART and a minimum of 2 VL measurements after 6 months of ART. To assess for concordance between immunologic and/or virologic failure, both CD4 cell count and VL were required at or after the point of failure by either method.
Patient data were collected on standardized clinic forms and entered into a customized electronic record database (FileMaker Pro) by trained data staff at each site. Baseline evaluations included medical history, physical examination, WHO clinical staging, complete blood counts, CD4 cell count, and plasma VL. These same clinical and laboratory evaluations were performed 3 and 6 months after ART initiation, then approximately every 6 months thereafter unless symptoms required more frequent monitoring.
All laboratory tests were performed on site in Nigeria. CD4 cell count measurement was performed using laser-based CD4 T-lymphocyte enumeration (Cyflow, Partec). Plasma HIV-1 RNA polymerase chain reaction determination was performed using the Roche Cobas Amplicor Monitor assay, version 1.5. All laboratories participate in regular external quality-control programs for HIV infection diagnosis, CD4 cell enumeration, and plasma VL estimation.
Virologic failure was primarily defined as 2 consecutive VL measurements >1000 copies/mL after at least 6 months of ART. Because this definition is used as the programmatic threshold for second-line switch, it is the primary outcome measure used in this study (ie, protocol-defined virologic failure). However, the recently revised WHO guidelines recommend using a definition of persistent VL >5000 copies/mL to define virologic failure ; thus, secondary analyses were performed using this as an alternative failure definition (ie, WHO-defined virologic failure). For patients with virologic failure, the time to failure was defined as the time from ART initiation to the first of 2 consecutive VL >1000 copies/mL.
Immunologic failure definitions were consistent with WHO guidelines , including (1) decrease in CD4 cell count to pretherapy baseline level (or below), (2) 50% decrease from the peak value during treatment, or (3) persistent CD4 cell counts <100 cells/mm3 after at least 12 months of ART. Because infection may cause transient CD4 cell count decrease, these failure definitions are intended for patients without concomitant infection. Therefore, individuals with a diagnosis of tuberculosis or other stage 3 or 4 infections (excluding oral candidiasis) within 6 months before or after CD4 cell count failure were not considered to have experienced immunologic failure in this study. Concurrent illness did not impact diagnosis of virologic failure.
On the basis of WHO guidelines, a single CD4 cell count meeting immunologic failure criteria was necessary to identify treatment failure. Because the APIN protocol required 2 consecutive VLs >1000 copies/mL to define ART failure, the performance of confirmatory CD4 cell count failure was also assessed. Specifically, secondary analyses were performed using a modified immunologic failure definition requiring 2 consecutive CD4 cell count measurements meeting the same or any of the 3 WHO CD4 cell count failure definitions.
Failures in this study, either by immunologic or virologic criteria, did not distinguish between those resulting from poor adherence and from drug resistance.
Statistical analyses were performed using Stata, version 10.1 (StataCorp). With use of virologic failure as the comparator, sensitivity, specificity, and predictive values (with 95% confidence intervals [CIs]) were calculated. For patients diagnosed with immunologic failure, either with or without concurrent virologic failure, the numbers of individuals identified as having immunologic failure by each of the 3 WHO immunologic failure definitions were compared using a χ2 test; a P ≤.05 was considered statistically significant. The Kaplan–Meier method was used to describe the cumulative proportion of individuals experiencing failure either by immunologic or virologic criteria. For those patients who did not meet failure criteria, follow-up data were censored at the time of the last available VL or CD4 cell count, depending on the specified group. For the subgroup of patients who experienced failure by both CD4 cell count and VL criteria, median time to failure was compared using a Wilcoxon signed-rank test for correlated samples.
All patients enrolled in the Harvard PEPFAR/APIN program provided consent for care. The treatment protocol was approved by the institutional review board (IRB) of the Harvard School of Public Health and the IRBs at Jos University Teaching Hospital, the National Institute of Medical Research, 68 Military Hospital, and University of Maiduguri Teaching Hospital. The Nigerian IRBs were registered with the US Federal Wide Assurance.
A total of 9690 patients enrolling from December 2004 through March 2008 met the protocol eligibility criteria, with 97.0% starting NNRTI-based ART and the remainder starting protease inhibitor–based or triple-nucleoside therapy primarily because of NNRTI intolerance. Baseline patient characteristics are summarized in Table 1. Sixty-four percent were female, median age was 35 years (interquartile range [IQR], 29–41 years), baseline CD4 cell count was 140 cells/mm3 (IQR, 77–203 cells/mm3), and VL was 53 162 copies/mL (IQR, 10358–181580 copies/mL).
The median duration of follow-up was 33.2 months (IQR, 24.8–40.5 months), resulting in 26 607 person-years of patient data. The median number of VL measurements per patient was 7 (IQR, 5–8), and median number of CD4 cell count measurements was 7 (IQR, 6–9), resulting in a mean of 1 CD4 cell count and VL evaluation every 4.7 months. In this dataset, there was excellent concordance of CD4 cell count and VL measurements. Of 73 644 laboratory entries, 87% contained both CD4 cell count and VL measurements for an individual patient on a specific date, 11.5% contained CD4 cell count but no VL from the same date, and 1.5% contained VL but no corresponding CD4 cell count.
With use of protocol-defined criteria, virologic failure was identified in 2097 (21.6%) of 9690 individuals (Table 2), and 1329 (13.7%) met WHO-defined VL failure criteria. In resource-rich settings, virologic failure is often defined as a confirmed VL >200 or >50 copies/mL. Although we chose to use the programmatic definition of virologic failure as the primary comparator in this analysis, using a cutoff of 400 copies/mL (the lower limit of detection for the assay in the program) would have increased the proportion experiencing failure to 28.3%.
Overall, 3122 (32.2%) of 9690 patients met any of the immunologic failure criteria. The proportion of individuals identified by each of the 3 or a combination of immunologic failure definitions is shown in Table 4. Because protocol-defined virologic failure required 2 consecutive VLs >1000 copies/mL, we also analyzed rates of failure according to immunologic criteria if consecutive CD4 cell count confirmation was required (Table 2). When confirmation with either the same or any of the 3 immunologic definitions was required, the proportion of patients with immunologic failure decreased to 10.3% and 10.5%, respectively. Compared with standard WHO CD4 cell count failure criteria, significantly fewer individuals were identified as having immunologic failure when a modified definition of 2 consecutive CD4 cell count measurements was used (P < .0001 for both comparisons). Additional laboratory findings at the time of protocol-defined failure are displayed in Table 5. Of note, 48% of individuals were virologically suppressed (VL ≤400 copies/mL) at the time or within 6 months of immunologic failure.
Overall, 1225 of 9690 (13%) patients met both immunologic and virologic failure criteria. For patients with both virologic and immunologic failure, VL criteria identified failure significantly earlier (median, 10.4 months; IQR, 6.9–15.9 months) than did CD4 cell count criteria (median, 15.6 months; IQR, 10.0–23.5 months; P < .0001). The overall proportion of patients experiencing failure by any of the immunologic and/or virologic criteria among the 9690-patient cohort is displayed in the Figure 1. In this survival analysis, time to failure is compared between immunologic and virologic monitoring methods among the entire 9690-patient cohort. Patients were censored if they reached the protocol-defined end point of failure, were switched to a protease inhibitor–containing regimen, or, if at the point of last CD4 cell count or VL assessment, they did not meet failure criteria throughout follow-up.
With use of protocol-defined virologic failure as the gold standard, the sensitivity of immunologic criteria to detect viral failure was 58.4% (95% CI, 57.4%–59.4%), specificity was 75.0% (95% CI, 74.1%–75.9%), positive predictive value (PPV) was 39.2% (95% CI, 38.2%–40.2%), and negative predictive value (NPV) was 86.7% (95% CI, 86.0%–87.4%). When confirmation of CD4 cell count failure was required, either with the same or any of the 3 definitions on consecutive measurements, the sensitivity decreased to approximately 28% and specificity increased to 95% (Table 2). With use of WHO-defined virologic failure as the gold standard to assess immunologic criteria (Table 3), the sensitivity was 66.2% (95% CI, 63.6%–68.7%), specificity was 73.2% (95% CI, 72.2%–74.1%), PPV was 28.2% (95% CI, 26.6%–29.8%), and NPV was 93.2% (95% CI, 92.5%–93.8%).
To further evaluate the performance of each of the individual immunologic failure definitions, the number of patients meeting each of the 3 definitions and/or multiple definitions was determined (Table 4). An overall comparison of the distribution of immunologic definitions was significantly different between those patients meeting both immunologic and virologic failure criteria (true positives) and those meeting only immunologic failure criteria (false positives; P < .0001). Specifically, CD4 cell count failure definitions 1 and 2 were more commonly associated with false positives, whereas definition 3 was more commonly associated with true positives (P < .0001 for all comparisons). In addition, a significantly higher proportion of true positives (55.5%) than false positives (33.7%) met ≥2 immunologic failure definitions (P < .0001).
This is the largest cohort study to date demonstrating that WHO immunologic failure criteria lack sensitivity for predicting virologic failure. Almost 50% of individuals identified as experiencing failure by CD4 cell count criteria were virologically suppressed (VL, <400 copies/mL), whereas the criteria did not identify nearly half of individuals who were actually experiencing failure.
The conclusion that CD4 cell count criteria perform poorly remains true even when the slightly less stringent WHO definition of virologic failure is used (Table 3), but most patients in RLS are monitored using CD4 cell count or clinical criteria. Previous studies have shown a similarly low performance of immunologic and clinical criteria [12, 13]. However, this analysis, which uses programmatic treatment data, a large sample size, longer duration of follow-up, and rates of virologic failure commensurate with other real-life treatment cohorts in RLS [18–20], provides the strongest evidence to date of the poor performance of immunologic criteria in identifying treatment failure.
As a retrospective analysis of a large treatment cohort, this study is limited by the data available, the frequency of programmatic monitoring, and the program protocol for ART switch. Because virologic failure typically precedes immunologic failure in the natural progression of disease, this may have resulted in lower than expected sensitivity and PPV. However, given the risk of accumulating drug-resistance mutations in the context of nonsuppressive regimens, timely identification of failure remains critical for appropriate ART management. In addition, to compare monitoring strategies, a minimum number of laboratory values were required, as delineated in the inclusion criteria. Exclusion of patients with the poorest protocol compliance may have resulted in some underestimation of overall failure rates.
ART regimens that do not achieve virologic suppression are associated with an increased risk of disease progression and death , and identification of antiretroviral failure and prompt switching to second-line therapy may reduce the development of resistance [22, 23]. In this study, immunologic criteria did not detect 42% of failures identified by VL testing. Although rates of adverse events vary, a study found that 20% of patients experience serious adverse outcomes by 30 months with a regimen that did not achieve virologic suppression . Thus, a substantial number of unnecessary adverse outcomes would be expected if CD4 cell count monitoring alone were relied on to assess ART success.
The PPV modestly increased to 58.1% when confirmation of CD4 cell count criteria in 2 consecutive measurements was required (Table 2). As expected, requiring confirmatory CD4 cell count assessment resulted in a significant reduction in sensitivity. Less than one-quarter of individuals with virologic failure would have been identified if confirmation of CD4 cell count failure criteria had been required. Of note, although confirmation of CD4 cell count criteria is not necessary to meet WHO-defined immunologic failure criteria, there is a tendency for caregivers to use this method in practice (ie, checking a second CD4 cell count before making an ART switch decision). Clinicians should therefore be aware that, although specificity is considerably improved, a significant majority of treatment failures would be missed by this method.
In addition to potentially resulting in unnecessary adverse events, nonsuppressive regimens also contribute to the development of drug resistance. The number of resistance mutations has been found to correlate with duration of ART exposure [24–26]. In a study that evaluated patients who received virologically nonsuppressive combination ART over a median of 6 months, patients developed a mean of 1.96 drug resistance mutations (International AIDS Society), with a loss of 1.25 active drugs . There is also evidence that prolonged exposure to failing NNRTI regimens may compromise future treatment options, in particular, the use of etravirine [28–30]. As this study shows, relying on CD4 cell count criteria to diagnose treatment failure will, at best, result in a diagnosis significantly later than by VL monitoring (P < .0001). Perhaps more worrisome, this method misses nearly half of the failure cases, allowing for the selection of increasingly drug-resistant viruses. This raises the very real concern that subsequent second-line regimens may not perform as well because of compromise of nucleoside backbones as increasing class-wide resistance develops [22, 23, 31].
Among patients identified as experiencing failure by both immunologic and virologic criteria, VL monitoring identified failure significantly earlier than did CD4 cell count criteria (P < .0001). Thus, even among patients correctly identified as experiencing failure by CD4 cell count criteria, the potential exists for accumulated drug resistance mutations, given an increased time to failure. If more frequent VL monitoring recommendations had been used in the present study cohort, as is the case in resource-rich countries, it is likely that viral failure would have been identified even earlier.
Despite dramatic price reductions for ART, second-line ART currently remains almost 5 times more costly than first-line regimens. Along with negotiations to reduce drug prices, VL assays have also become more economical, with a mean cost of US $22 per VL test in the Harvard PEPFAR program. Discussions regarding effective monitoring strategies should therefore also take into consideration the potential impact on overall drug costs. In this study, the PPV was low, suggesting that less than half of patients identified as experiencing failure by CD4 cell count criteria were actually experiencing failure. Therefore, if CD4 cell count failure criteria were used to identify patients for switch, 60.8% (1897 of 3122 (Table 2) patients switched to second-line therapy would have been unnecessarily switched. Although further detailed cost analyses are underway, the significance of this should be noted, because 1897 patients unnecessarily switched to second-line therapy would cost in excess of US $1 million in increased treatment costs per year.
Immunologic criteria not only misclassified a significant number of patients as failures, but also identified a significantly larger number of treatment failures overall. In this cohort of 9690 patients, immunologic criteria identified 3122 failures (32.2%), and virologic criteria identified only 2097 (21.6%; P < .0001). The incremental increase in cost resulting from a greater number of identified failures should also be considered when assessing the overall value of CD4 cell count versus VL monitoring.
In conclusion, this large cohort study shows that immunologic criteria are poor predictors of virologic failure, missing nearly half of individuals who were failing ART and misidentifying nearly half of patients with immunologic failure who were actually virologically suppressed. The impact of accumulated drug resistance and unnecessary drug switches may ultimately eclipse the cost of VL monitoring and potentially erode the gains achieved through widespread ART use. Although the development of point-of-care HIV RNA quantification may alter the feasibility discussion, suitability for high-volume urban clinics remains to be seen. As HIV treatment programs in RLS progress beyond the emergency phase, commitments to building the infrastructure for optimal patient monitoring may improve patient outcomes and long-term sustainability.
This work was funded in part by the US Department of Health and Human Services, Health Resources and Services Administration (U51HA02522-01-01). The contents are solely the responsibility of the authors and do not represent the official views of the funding institutions.
All authors: No reported conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.