|Home | About | Journals | Submit | Contact Us | Français|
(See the editorial commentary by Branson and Stekler, on pages 521–4.)
Background.Most human immunodeficiency virus (HIV) point-of-care tests detect antibodies (Ab) but not p24 antigen (Ag) or RNA. In the absence of antibodies, p24 antigen and RNA typically indicate acute HIV infection. We conducted a field evaluation of the Determine® HIV-1/2 Ag/Ab Combo rapid test (Combo RT).
Methods.The antigen portion of the Combo RT (for acute HIV infection) was compared with a Roche Monitor HIV RNA polymerase chain reaction assay. The antibody portion of Combo RT (for established HIV infection) was compared with rapid test algorithms. Participants were enrolled at a sexually transmitted infection clinic and HIV testing and counseling center in Lilongwe, Malawi. Rapid testing was conducted with parallel testing in the clinic and serial testing in the center. The Combo RT was performed in clinic participants with negative or discordant antibody results and in all center participants.
Results.Of the participants 838 were HIV negative, 163 had established HIV infection, and 8 had acute HIV infection. For detecting acute HIV infection, the antigen portion had a sensitivity of 0.000 and a specificity of 0.983. For detecting established HIV infection, the antibody portion had a sensitivity of 0.994 and a specificity of 0.992.
Conclusions.Combo RT displayed excellent performance for detecting established HIV infection and poor performance for detecting acute HIV infection. In this setting, Combo RT is no more useful than current algorithms.
Point-of-care rapid tests for human immunodeficiency virus (HIV) antibody (Ab) detection have facilitated the scale-up of HIV counseling and testing throughout sub-Saharan Africa [1, 2]. The sensitivity of these tests approaches 100% for antibody detection [3, 4]. However, the tests cannot identify persons with acute HIV infection who have not yet developed HIV-specific antibodies [5–7]. Persons with acute HIV infection are often hyperinfectious because of high viral loads [8–12]. Integrating acute HIV infection detection into HIV testing algorithms would enable acutely infected persons to learn their true HIV status, rather than being informed that they were HIV seronegative. Identifying these persons with acute HIV infection could enable intervention to prevent transmission and early treatment, potentially preserving immune function [13, 14].
Identification of acute HIV infection requires detection of HIV nucleic acids or p24 antigens. Available assays are laboratory based, resource intensive, and require follow-up. HIV RNA polymerase chain reaction (PCR), used for either individual or pooled samples, is the reference standard for detecting antibody-negative acute HIV infection, but it is expensive and difficult to implement in resource-poor settings. HIV p24 antigen (Ag) enzyme-linked immunosorbent assays (ELISAs) have good performance characteristics compared with HIV RNA PCR analysis, but they have been challenging to implement on a wide scale. Fourth-generation HIV ELISAs detect both antibodies and antigens [6, 15, 16] but do not distinguish between the two and require venipuncture, a laboratory, and patient follow-up, limiting routine use in most settings.
A rapid point-of-care test capable of distinguishing established from acute HIV infection could improve the sensitivity of existing algorithms and enable provision of acute HIV infection results in real time . The Determine® HIV-1/2 Ag/Ab Combo (Combo RT) is a point-of-care rapid test with separate indicators for HIV antibodies and p24 antigen. The Combo RT was designed to identify HIV earlier than other conventional rapid tests. The antibody portion is reported to be analogous to the Determine® HIV-1/2 antibody test, a widely used rapid test for HIV identification. The antigen component of the test is intended to expand the diagnostic spectrum to identify persons with circulating free p24 antigen, unbound to antibodies. During development, Combo RT antigen was assessed using stored serum from commercial seroconversion panels . For primary HIV samples in the pre- or periseroconversion period, the reported sensitivity of the antigen portion of the Combo RT was 92.2%, compared with a fourth-generation HIV ELISA as the reference standard. Specificity of the antigen portion of the test was reported at 96.6%. The Combo RT is currently commercially available outside the United States.
We conducted a field evaluation in Lilongwe, Malawi, to assess the accuracy of the antigen portion of Combo RT to detect persons with acute HIV infection. The Roche Monitor HIV RNA PCR assay was used to identify persons with acute HIV infection after routine HIV rapid test evaluation for established HIV infection. We also performed an “ultrasensitive” heat-dissociated p24 antigen ELISA. Finally, in a subset of the study population, we assessed the antibody portion of Combo RT against a standard rapid test antibody algorithm.
Participants were recruited from the Kamuzu Central Hospital Sexually Transmitted Infection Clinic and the Lighthouse Trust HIV Testing and Counseling Center in Lilongwe, Malawi, between October 2009 and June 2010. At both sites, patients were eligible if they were ≥18 years old, and willing to provide up to 1200 μL of blood in microvette capillary tubes, and willing to provide informed consent. In the HIV testing and counseling center, all consenting persons presenting for HIV testing were enrolled before standard opt-out HIV rapid testing. In the sexually transmitted infection clinic, only patients with negative or discordant HIV rapid test results at high risk for acute HIV infection were invited to participate to minimize the number of persons screened and maximize the likelihood of identifying persons with acute HIV infection . Participants completed an interviewer-administered questionnaire on demographics, sexual behavior, and symptoms.
All personnel conducting the standard HIV rapid tests and the Combo RT were highly experienced HIV counselors. Training in the performance and interpretation of the Combo RT was conducted at the initiation of the study. Procedures were reviewed during weekly study meetings. All tests were performed according to the manufacturer instructions, including reading results within the specified time frame.
The Combo RT (Inverness Medical Japan) is an in vitro, visual qualitative immunoassay for use with human serum, plasma or whole blood . For whole blood, 50 μL is required followed by a chase buffer. If antibodies are present, they bind to an antigen–selenium colloid and recombinant antigen in the test strip to form a line. If free p24 antigen is present, it binds to a biotinylated anti-p24 antibody in the sample pad, and to selenium colloid anti-p24 antibody and immobilized avidin in the test strip to form a separate line.
To determine whether p24 antigen was present, an ELISA (Alliance HIV-1 p24; Perkin Elmer Life Sciences) was used with additional steps of heat dissociation and signal amplification. Specimens were lysed, boiled to disrupt antigen–antibody complexes, and enhanced with signal amplification (ELAST; Perkin Elmer Life Sciences) [20, 21]. These steps have been followed previously in this setting to improve sensitivity of p24 antigen detection [22, 23].
In the sexually transmitted infection clinic, routine parallel testing was performed with Determine® HIV-1/2 (Inverness Medical Japan) and Uni-Gold™ Recombigen® HIV (Trinity Biotech). As a part of routine procedures in this clinic, persons with negative or discordant HIV antibody rapid test results and a high risk score on a validated acute HIV risk score algorithm were referred for individual testing with a Roche Monitor HIV RNA PCR assay (Roche Molecular Systems) . After informed consent was obtained, specimens were obtained by finger stick for the Combo RT, which was performed on site, and the ultrasensitive p24 antigen assay, which was performed in the laboratory.
In the HIV testing and counseling center, routine serial testing was performed first with the Determine® HIV-1/2 assay, and then, if positive, with the Uni-Gold™ Recombigen® HIV assay following national protocol. For discordant results, SD Bioline HIV 1/2 3.0, (Standard Diagnostics) was the arbiter. As part of study protocol, the Combo RT was added for study participants. All study participants whose results were negative or discordant (positive for Determine® HIV-1/2 antibody assay, negative for Uni-Gold™ Recombigen® and Bioline HIV 1/2 3.0 assays) were tested individually with the Roche Monitor HIV RNA PCR and ultrasensitive p24 antigen assay.
In both settings, participants were classified as HIV negative if they had negative or discordant rapid antibody test results and undetectable HIV RNA. They were classified as having acute HIV infection if they had negative or discordant rapid antibody test results and detectable HIV RNA and were classified as having established HIV infection if they had 2 positive rapid antibody test results.
Among individuals without established HIV infection, the sensitivity and specificity of the Combo RT antigen (Ag) and the ultrasensitive p24 antigen assay were calculated using the definition of acute HIV infection mentioned earlier. Additionally, among those without established HIV infection, the sensitivity and specificity of the Combo antibody alone, and in combination with Combo RT Ag, were assessed for acute HIV infection using the foregoing definition. Among those without acute HIV infection, the performance of Combo RT antibody (Ab) was assessed for established HIV infection using the definition of established HIV infection mentioned earlier. We used exact methods to calculate 95% confidence intervals (CIs). The κ coefficient for agreement between Combo RT Ab and Determine® HIV-1/2 results was calculated. Analyses were conducted using SAS software, version 9.2 (SAS Institute) and Stata software, version 10 (StataCorp).
The study was approved by the Institutional Review Board at the University of North Carolina at Chapel Hill and by the National Health Sciences Research Committee, which is the local ethics committee in Malawi.
Overall, 1009 participants had complete HIV status information and a Combo RT Ag result, including 195 in the sexually transmitted infection clinic and 814 in the HIV testing and counseling center (Table 1). An additional 5 persons in the sexually transmitted infection clinic and 29 in the HIV testing and counseling center were excluded because their true HIV status could not be determined or because they lacked a Combo RT Ag result.
Most participants were male (sexually transmitted infection clinic, 65%, HIV testing and counseling center, 52%). The mean age was 28 years (standard deviation, 7) in the sexually transmitted infection clinic and 30 years (standard deviation, 9) in the HIV testing and counseling center. In the sexually transmitted infection clinic, 7 persons (4%) had acute HIV infection and 188 (96%) were HIV negative. In the HIV testing and counseling center, 1 person (0.1%) had acute HIV infection, 163 (20%) had established HIV infection, and 650 (80%) were HIV negative.
In the field, using whole blood, results of the Combo RT Ag were negative in all 8 persons with acute HIV infection (sensitivity, 0.000; 95% CI, .000–.369) (Table 2). HIV RNA values for the 8 persons with acute HIV infection were ≥750000 copies/mL in 6 and 45000 and 359000 copies/mL in the other 2. After completion of enrollment, we examined stored plasma from 6 of the persons with acute HIV infection; all 6 of the stored plasma specimens were negative with the Combo RT Ag.
Among the 838 uninfected persons across both testing sites, we observed 14 false-positive Combo RT Ag results, for an overall specificity of 0.983 (95% CI, .972–.991), a specificity of 180/188 or 0.957 (95% CI, .918–.982) for the sexually transmitted infection clinic, and a specificity of 644/650 or 0.991 (95% CI, .980–.997) for the HIV testing and counseling center. None of the HIV-uninfected persons with false-positive Combo RT Ag results had discordant rapid test results.
With the observed sensitivity and specificity, the positive predictive value (PPV) of the Combo RT Ag would be 0%, regardless of acute HIV prevalence. However, given that the sensitivity estimate is based on a small number of specimens, we assessed the upper range of plausible PPVs of the Combo RT Ag using the upper 95% confidence limits of the overall sensitivity (0.369), the overall specificity (0.991), and the specificity in the HIV testing and counseling center (0.997). With the overall sensitivity and specificity values, in a setting with a relatively high acute HIV prevalence (1%), the PPV would be 29.3%, and with the HIV testing and counseling center specificity, the PPV would be 55.4%.
Of 8 persons with acute HIV infection, 7 had sufficient specimen quantity for ultrasensitive p24 antigen assessment. Five of 7 persons with acute HIV infection had detectable p24 antigen with this assay (sensitivity, 0.714; 95% CI, .290–.963). The HIV RNA concentrations for the 2 persons with negative ultrasensitive p24 antigen assays were 45000 and ≥750000 copies/mL. No false-positive results were obtained with the ultrasensitive p24 antigen assay (806/806; specificity, 1.000; 95% CI, .995–1.000).
Although the Combo RT Ag results were negative in all 8 persons with acute HIV infection, 2 persons without established HIV infection had positive Combo RT Ab results (sensitivity, 0.250; 95% CI, .032–.651) (Table 2). However, 6 of 836 persons without HIV infection had false-positive antibody results (specificity, 0.993; 95% CI, .984–.997), and 18 of 836 persons had false-positive antibody or antigen results (specificity, 0.979; 95% CI, .966–.987). In a setting with a 1% acute HIV prevalence, the PPV for acute HIV detection among persons without established HIV infection would be 26.1% for the antibody strip alone and 10.5% when the antigen and antibody strips were considered together.
Of the 813 participants from the HIV testing and counseling center without acute HIV infection, 2 HIV-negative persons had missing Combo RT Ab results. Of 811 HIV testing and counseling participants evaluated with Combo RT Ab, 163 had established HIV infection using the HIV testing and counseling center algorithm. The antibody portion of the Combo RT identified 162 of these persons (sensitivity, 0.994; 95% CI, .966–1.000). Among 648 persons without HIV infection, we observed 5 false-positive Combo RT Ab results, yielding a specificity of 0.992 (95% CI, .982–.997) (Table 2). In this population with an established HIV infection prevalence of 20%, the PPV of the antibody portion was 0.970 (95% CI, .932–.990).
We also compared the Combo RT Ab to the standard Determine® HIV-1/2 Ab test used routinely in the HIV testing and counseling center. Among 171 persons with positive Determine® HIV-1/2 Ab results, 165 had positive and 6 had negative Combo RT Ab results. Among 640 persons with negative Determine® HIV-1/2 Ab results, 638 had negative and 2 had positive Combo RT Ab results. The κ coefficient for overall agreement was 0.970 (95% CI, .950–.991).
Identification of persons with acute HIV infection represents a significant challenge, owing to the absence of antibodies in the earliest stages, limitations of standard rapid tests to detect p24 or HIV RNA, and logistical and cost issues with p24 antigen and HIV RNA assays. However, given the critical role of acute HIV infection in HIV transmission, an acute HIV test meeting the “ASSURED” criteria (Affordable, Sensitive, Specific, User-friendly, Robust and rapid, Equipment-free, and Deliverable)  for routine worldwide use is urgently needed. The Combo RT has been marketed as such a test, because it is relatively inexpensive and can be administered at the point of care. However, in this field evaluation of the Combo RT, the sensitivity and specificity of the p24 antigen component of the test were inadequate for widespread use for detecting acute HIV infection. The Combo RT Ag failed to detect the 8 cases of acute HIV infection and falsely classified 14 other persons with acute HIV infection. However, 2 cases were identified with the antibody component.
Developing a point-of-care test with adequate sensitivity and specificity for acute HIV detection is extremely challenging. Given the brief window of acute HIV infection, the acute HIV prevalence in a population is very low at any given point in time, even among relatively high-risk groups. Consequently, a test must have exceptional performance characteristics to be useful, especially without additional confirmatory testing. The Combo RT was designed to provide a marginal reduction in the HIV window period. However, even assuming that the sensitivity of the Combo RT for acute HIV infection was 50%, in a population with a high acute HIV prevalence (1%), more than half of those identified with acute HIV infection would be false positives, unless the specificity was greater than 0.995. Thus, to detect even 50% of cases with reasonable certainty, a near-perfect specificity is required, a characteristic not achieved in our field evaluation.
Evaluation of tests capable of identifying persons with acute HIV infection is also complicated by the low prevalence of acute HIV infection and the brief window period. During development, the Combo RT was evaluated using stored specimens to ensure a reasonable number of cases of acute HIV infection could be assessed . However, evaluation in a laboratory with serum, plasma, or spiked whole blood is not the same as a field evaluation, especially for a test designed to be used at the point of care. Field evaluations on finger-stick blood are crucial for assessing real-world performance characteristics, even though they are expensive, time consuming, and challenging to conduct. Although the performance characteristics of the Combo RT Ag were insufficient for use in this clinical setting, field evaluation in other settings may be warranted to assess whether the test performs better.
The Combo RT has been previously evaluated with stored serum or plasma specimens. In a comparison with the laboratory-based ARCHITECT HIV Ag/Ab Combo (Abbott Laboratories) Combo RT detected fewer acute infections in stored serum or plasma (17% vs 80%) and fewer recent infections (59% vs 89%) on a panel of specimens . In a clinic-based performance assessment of 5 rapid tests, neither the antigen nor the antibody component of Combo RT detected the 2 recent infections using finger-stick blood .
The reasons for low sensitivity of the Combo RT Ag in our field evaluation may be related to the time course and detection of p24 antigenemia. The heat-dissociation and signal-amplification steps used in the ultrasensitive p24 antigen assay substantially improve the detection of the p24 antigen, which is often in a bound form [20, 22, 23]. The ultrasensitive p24 antigen assay provides the most sensitive antigen detection method available. The limit of detection of the ultrasensitive p24 antigen assay can be as low as 172 fg/mL , ~100 times lower than the limit of detection reported for the Combo RT antigen component (12.5–25 pg/mL) . However, the heat-dissociation and signal-amplification steps are not feasible for a rapid-test strip. In some cases, HIV RNA may be detectable through PCR, but p24 antigen levels may be below detection limits, even with ultrasensitive p24 antigen assays. In our study, among the 7 persons with acute HIV and ultrasensitive p24 antigen assay results, 2 were negative for p24 antigen, suggesting very low levels of antigenemia. Of those with p24 antigen present in the ultrasensitive assay, the concentration of free, unbound p24 antigen may have been below the detection limits of the Combo RT Ag, other non–heat-dissociated p24 assays, or both.
Alternatively, performance could have been due to reduced ability to detect clade C HIV. Although we did not assess subtype in this study, clade C is the predominant Malawian strain and accounts for nearly all HIV infections [28–30]. The package insert and published data to date provide little information about the performance of the antigen component of the Combo RT by clade [18, 31, 32]. Among 22 stored plasma samples from persons with clade C acute HIV infection, results for the antigen portion of the Combo RT were negative in all 22, although 6 had positive results with the antibody portion (J Moodley, unpublished data).
One potential concern regarding the Combo RT performance is the incorporation of the Determine® HIV-1/2 antibody test in the rapid test algorithm used in the reference standard. Although both the Determine® HIV-1/2 antibody test and the Combo RT use the same platform, the false-positive antigen results with the Combo RT cannot be explained by an association with false-positive Determine® HIV-1/2 antibody results in the reference standard. None of the false-positive Combo RT Ag results were observed in persons with discordant rapid test results (ie, positive for Determine® HIV-1/2 antibody and negative for Uni-Gold™ Recombigen® HIV antibody).
In contrast to the antigen component, the antibody portion of the Combo RT performed well in detecting established HIV infection, consistent with the performance of the Determine® HIV-1/2 antibody test [3, 4]. The strong correspondence of the Combo RT antibody component and the Determine® HIV-1/2 rapid test suggests that the Combo RT was deployed properly in the field.
The antibody component of Combo RT also identified more persons with acute HIV infection than the antigen component. Acute HIV infection was not detected by the antigen component in any participant but was detected by the antibody component in 25% of participants. The sensitivity of the antibody component for acute HIV detection is compatible with results from similar settings, where discordant antibody results are strong predictors of acute HIV infection [19, 33]. In a setting with a 1% acute HIV prevalence, in a population of persons without established HIV infection, a positive result on either of the 2 Combo RT strips is indicative of acute HIV infection 10% of the time, but a positive result on the antibody strip alone is indicative of acute HIV infection 26% of the time. This observation underscores the excellent performance characteristics of the antibody component of the Combo RT, and the poor, even potentially detrimental, characteristics of the antigen component.
Closing the seronegative window after HIV infection is important. Persons with acute HIV infection contribute disproportionately to the HIV epidemic , yet these cases are difficult to diagnose. A point-of care acute HIV diagnostic that meets all ASSURED criteria is urgently needed. Assessment of any prospective test for acute HIV infection must start with evaluation of stored specimens, commercial specimen panels, and spiked blood samples. However, at some point, assessment must include field evaluation to offer clinicians information on how to interpret results in clinical settings. This field evaluation suggests that a positive antigen report on the Combo RT is far more likely to be a false-positive result than a true case of acute HIV infection, a finding that must be considered if this test is used in practice.
We would like to acknowledge the hard work of the teams at the Lighthouse Trust HIV testing and counseling center, Kamuzu Central Hospital Sexually Transmitted Infection Clinic, and UNC Project laboratory. Specifically, we would like to thank Clement Mapanje, Happiness Kanyamula Malino, and Hilex Kamzati at the sexually transmitted infection clinic. We would like to thank Shadreck Puruma at the HIV testing and counseling center. We would also like to thank Inverness (now Alere) for a limited supply of test kits.
This study was supported by grants from the US National Institutes of Health [DHHS/NIH/NIAID 5 T32 AI 07001-34 and NIAID R01 AI083059]. Study sponsors played no role in study design, collection or analysis of data, interpretation of results, writing of the manuscript, or decision to submit for publication. Inverness (now Alere) provided 69 (6.8%) of the Determine® HIV-1/2 Ag/Ab Combo rapid tests used in this study.
S. A. F. has consulted for Abbott Molecular, Roche Molecular, and Gen-Probe. All other authors report no potential conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.