|Home | About | Journals | Submit | Contact Us | Français|
Herpes Simplex Virus-2 (HSV-2) is a risk factor for HIV-1 infection; HSV-2 serology may be useful for HIV-1 prevention. We characterized HSV-2 serology assay performance in HIV-positive and HIV-negative Africans.
Serostatus for HSV-2 and HIV-1 was determined in 493 serum specimens stored from a community HSV-2 prevalence survey in Kampala, Uganda. HSV-2 serology by Focus HerpeSelect ELISA, Biokit HSV-2 rapid assay, and Kalon HSV-2 were compared to Western Blot (WB) by HIV-1 serostatus.
Sensitivity/specificity were: 99.5%/70.2% for Focus, 97.0%/86.4% for Biokit, and 97.5%/96.2% for Kalon. Focus with Biokit confirmation improved sensitivity/specificity (99.4%/96.8%, respectively). Use of a higher Focus index value cutoff of 2.2 instead of 1.1 increased specificity from 70.2% to 92.4%. Kalon had higher specificity than Focus (p<.001).
Of commercially available HSV-2 serologic assays, Kalon alone, or Focus ELISA followed by Biokit confirmation perform best. Improved HSV-2 assays are needed for HSV-2 and HIV-1 public health activities in Africa.
New biomedical strategies to reduce acquisition and transmission of HIV-1 are needed to compliment existing strategies such as behavior change, condoms, and male circumcision. Over the last decade, genital herpes due to HSV-2 infection, has become a major focus for HIV-1 prevention. HSV-2 is the principal cause of genital ulcer disease (GUD) globally, including in Africa. Antiviral regimens can reduce the duration and frequency of herpes symptoms. The majority of HSV-2 reactivation is subclinical and unrecognized by infected individuals and their clinical providers [1–4], thus serology tests are needed for public health initiatives that require identifying the vast majority of persons with unrecognized HSV-2 infection.
Evidence is accumulating for a synergistic interaction between HSV-2 and HIV-1 [5, 6]. Observational data indicate that approximately 85% of HIV-infected persons in Africa are co-infected with HSV-2, and that HSV-2 increases HIV-1 infectiousness and is associated with increased risk of HIV-1 transmission to sexual partners [5, 7–9]. Furthermore, among HIV-1 infected persons, HSV-2 recurrence rate, and the severity and duration of symptoms are increased in persons with CD4<200 [5, 7, 10, 11]. On the other hand, HSV-2 seroprevalence is 50% among HIV-uninfected and is associated with 2–3 fold increased risk of HIV-1 acquisition [5, 7, 12].
Although recent clinical trials using acyclovir for HSV-2 suppression in HIV-negative, HSV-2 seropositive persons did not show a reduction in HIV-1 acquisition [13, 14], multiple placebo-controlled, double-blind proof-of-concept trials have demonstrated that HSV-2 antiviral suppression does reduce HIV-1 plasma and genital viral load [15, 16]. This suggests that HSV-2 suppression may reduce HIV-1 transmission and disease progression providing an option for delaying initiation of antiretrovirals in persons with intermediate CD4 counts. An ongoing clinical trial in Africa in HIV-1 discordant couples is comprehensively evaluating this concept [7, 17].
Two “gold standard” type-specific serologic assays, Western Blot (“WB”) and monoclonal antibody inhibition assays [18, 19], have been developed based on detection of antibody response to the type-specific gG herpes virus glycoprotein. Several gG-based type-specific commercial serology assays have been developed and some approved by US Food and Drug Administration (FDA) after testing against the gold-standard WB [19, 20]. One of these, the HerpeSelect HSV IgG ELISA (“Focus”; Focus Technologies, Cypress, CA, USA), is widely used both in the United States and worldwide. The specificity of the test varies by regional background with higher false positive rates among African compared to American Caucasian cohorts [20–22]. A second HSV type 2 IgG ELISA (“Kalon”; Kalon Biological Ltd, Surrey, UK; not FDA approved) has been shown to have superior sensitivity to Focus in African sera [20, 21, 23]. A third FDA approved, type-specific assay, Biokit HSV-2 Rapid Assay (“Biokit”; Biokit USA, Lexington, MA, USA; formerly POCkit-HSV-2 from Diagnology), also has high sensitivity and specificity [19, 20, 24]. Biokit can improve the positive predictive value of the Focus assay when used to confirm positive specimens in North American cohorts [19, 24, 25].
Given the performance differences of commercial HSV-2 assays in African populations, additional data are needed to inform recommendations for use of HSV-2 serologic assays in the context of future HIV-1 prevention activities. In this study we tested specimens collected in an Ugandan sero-surveillance study by Focus, Kalon and Biokit HSV-2 serologic assays using the type-specific HSV Western blot (WB) as a gold standard  to determine sensitivity and specificity of combination testing algorithms and use of different cut-offs of the EIA assays .
Sera were collected from 1126 adults in the Kawempe Division in Kampala, Uganda for a 2004 “Seroprevalence and incidence of genital herpes in Uganda” survey performed by the Department of Medical Microbiology, Makerere University, Kampala . Of 1124 individuals in this cohort with questionnaire data collected, 786 (70%) were female, 546 (48%) were less than 25 years of age and 599 (53%) were married. Local testing with 1st generation Focus yielded 657 positive (index >3.4), 193 low positive (index between 1.1 and 3.4) and 276 negative (index <0.9) results. A subset of stored samples were sent to Seattle including all low positive samples (n=193) and a subset of “high positive” or negative samples (n=150 for each category). These specimens were made anonymous before shipping to the University of Washington (UW) for more extensive HSV-2 and HIV-1 serology testing.
All 493 specimens sent from Kampala were evaluated at the University of Washington Virology Laboratory using commercial tests performed according to kit instructions; WB was run as previously described . “Equivocal” results were scored if bands on the HSV-2 blot were insufficient in number or intensity to qualify as a positive result after preabsorption against HSV-1 antigens. Technicians were blinded to HIV status and results of other assays; whether performed in Seattle or Kampala. Results were analyzed only after all testing was completed.
Focus testing in Seattle utilized a 2nd generation kit modified by the company to improve specificity. Focus 1st and 2nd generation and Kalon assays have the same manufacturer recommended index value cut-offs for test interpretation: index values <0.9 are negative, >1.1 are positive, and index values 0.9 –1.0, inclusive, are equivocal. Index values between 1.1 and 3.4 were classified for this study as “low positive” and >3.4 as “high positive”. Biokit test results were recorded as positive when the test spot was clearly colored red or pink, and negative when the test spot was faint or without color.
Sera were tested by three different HIV-1 ELISAs: Bio-Rad Genetic Systems ™ HIV-1/HIV-2 Plus O ELISA (Bio-Rad Laboratories, Redmond, WA, 98052), Abbott HIVAB™ HIV-1/HIV-2 (rDNA) ELISA (Abbott Laboratories, Abbott Park, IL. 60064),) and Vironostika® HIV-1 Microelisa System (bioMerieux, Inc, Durham, NC. 27704). Sera reactive by any single ELISA assay were confirmed utilizing the Bio-Rad Genetic Systems ™ HIV-1 WB (Bio-Rad Laboratories, Redmond, WA, 98052). For this analysis, samples with positive results on the HIV-1 WB were identified as HIV-1 positive. Samples were identified as HIV-1 negative if ELISA results were uniformly negative or the WB was negative after initial reactive or contradictory ELISA results.
Sensitivity and specificity of the different HSV-2 serological tests and combinations were computed with HSV-2 WB as the ‘gold standard,’ using manufacturer cut-offs for positivity. Positive and negative predictive values were computed for the observed HSV-2 prevalence and a range of possible population HSV-2 seroprevalences.
Samples evaluated in Seattle included all those with a Focus low positive test result in Kampala and a proportion of those from other Focus result categories (i.e. negative, equivocal and high positive) based on tested in Kampala. This represented a non-random selection of specimens from the original Kampala study set. We therefore inferred testing results for samples not tested in Seattle as being confirmed by WB and Biokit in the same proportions as tested samples within each Focus category. HSV-2 confirmation rates among the samples were estimated based on Focus testing in Kampala (using 1st generation Focus), subsequent agreement by Focus testing in Seattle (with 2nd generation Focus), and further confirmation of Seattle Focus testing by WB testing in Seattle. The corresponding uncertainty of these confirmation rates was estimated according to the number of results received. Confidence intervals for accuracy rates were computed using the normal approximation to the binomial. Interim confidence intervals were computed assuming low and high WB confirmation rates in untested samples using the anticipated variability based on tested samples. Final confidence intervals include uncertainty in both the number of samples confirmed and in the estimated confirmation rates of those not confirmed.
An analysis of Focus serology results was initially performed using the manufacturer’s index cutoff for Focus positivity described above. In a subsequent analysis, an alternate index cutoff for Focus positivity was optimized to this Ugandan study population using Receiver-Operator Curves (ROC)  with HSV-2 status by WB as the gold standard. For this analysis, Focus index values for each category of WB result for Kampala samples not tested in Seattle were simulated following the normal distribution and using mean and standard deviation of the observed Focus index values on the log10 scale. This was done to develop datasets representative of the population. Simulations for the 633 untested sera were performed three times and ROC curves were built for each "augmented" dataset. An alternate index value cutoff was chosen that maximized the sum of sensitivity and specificity by choosing the point nearest the top left corner of the ROC curve, thus treating false positives and false negatives equally. Confidence intervals for sensitivity and specificity were again computed using the new Focus cutoff, as well as updated positive and negative predictive values over a range of potential population prevalence rates. Focus test results reflecting the manufacturer’s index cutoff of 1.1 and results reflecting the ROC optimized cutoff are referred to as “Focus 1.1” and “Focus 2.2”, respectively.
ROC curves were created in a similar fashion for Kalon results to assess optimal cutoffs with the manufacturer’s and optimized cutoffs found to provide similar results. Therefore, manufacturer’s cutoffs for Kalon were used in all analyses.
McNemar's test was performed to compare sensitivity and specificity of Kalon versus Focus (using either index cutoff) and Focus with positive results confirmed by Biokit. Only samples tested by relevant methods in Seattle were used in these calculations; assumed rates in samples tested only in Kampala were not included. Since the samples sent to Seattle were heavily weighted with low positive results based on Kampala Focus findings, the accuracy measures apply to a population that is not representative; however, the comparison remains valid as it is based on completed tests and is preferable to assuming that confirmation rates would be similar to non-tested samples. All subjects regardless of HIV-1 status were included in the analysis, with the analysis repeated using only HIV-1 infected participants.
The Joint Clinical Research Centre (JCRC) and Case Western University Institutional Review Boards reviewed and approved the protocol for the Kampala seroprevalence survey. Subsequently, the JCRC, Uganda National Council for Science and Technology, and University of Washington IRBs approved HSV-2 and HIV-1 testing of anonymous specimens at the UW.
Among 493 specimens shipped to the UW, WB testing yielded 164 (33%) positive, 303 (62%) negative and 26 (5%) equivocal for HSV-2 antibodies. Focus testing in the University of Washington Virology laboratory resulted in 144 (29%) high positive, 121 (24%) low positive, 29 (6%) equivocal, and 199 (40%) negative samples; Biokit resulted in 141 (29%) positive, 1 (0.2%) equivocal and 351 (71%) negative; and Kalon demonstrated 171 (35%) positive, 10 (2%) equivocal and 312 (63%) negative (Table 1).
After excluding WB equivocal samples and adjusting proportions to account for sampling from the original community survey cohort, an ROC analysis of Focus testing results yielded an optimum positive index value cutoff of 2.2 (test results using this cutoff are referred to as “Focus 2.2”) for each augmented dataset instead of the manufacturer recommended cutoff of 1.1 (test results using this cutoff are referred to as “Focus 1.1”).
We evaluated sensitivity and specificity of HSV-2 assays individually and in combination relative to WB (Table 2). In this analysis, Focus 1.1 had the highest sensitivity, 99.5%, and lowest specificity, 70.2% and Biokit had the highest specificity, 97.0% and lowest sensitivity, 86.4%. Biokit confirmation of Focus 1.1 positive results improved Focus sensitivity and specificity to 97.5% and 96.2%. These figures are similar to results from Kalon testing which had 99.4% sensitivity and 96.8% specificity with the 1.1 cutoff. After optimizing the index value cutoff for this Ugandan population, Focus 2.2 maintained high sensitivity (96.4%), and had markedly increased specificity (92.4%) compared to Focus 1.1 (70.2%). Kalon’s higher specificity as compared with Focus 1.1 was statistically significant (p<0.001). Focus 1.1 had a higher sensitivity over Kalon that was borderline significant (p=0.063).
Biokit confirmation of Focus positive results, using either a Focus cut-off of 1.1 or 2.2, resulted in a substantial number of samples (111 using Focus 1.1+Biokit and 33 using Focus 2.2+Biokit) being deemed "indeterminate" and excluded from the analysis due to a Focus positive result being followed by a negative Biokit result. These samples were mainly among the low positives, so their exclusion may obscure the comparison between these tests.
Sensitivity and specificity values for each assay were used to calculate positive (PPV) and negative predictive values (NPV) as a function of HSV-2 prevalence (Figure 1). PPV declines for Focus 1.1 and Focus 2.2 at low HSV-2 prevalence. NPV was lowest for Biokit, intermediate for Kalon and Focus 2.2, and highest for Focus 1.1 +/− Biokit. NPV declines at high HSV-2 prevalence, particularly for Biokit, Kalon and Focus 2.2+Biokit; however, statistical comparisons were not performed for differences in predictive values since these values depend on accurate estimation of the seropositivity prevalence, something we could not confidently estimate given the non-random selection of this subset of samples.
We also evaluated HSV-2 serologic assay performance by HIV-1 serostatus. Among the 467 samples with definitive HSV-2 WB results, using HIV-1 WB as the gold-standard, we identified 399 (85.4%) HIV-1 seronegative, 57 (12.2%) HIV-1 seropositive and 11 HIV-1 WB indeterminate in this cohort (Table 3). Sensitivity and specificity of each HSV-2 assay for HIV-1-seropositive and HIV-seronegative cohorts generally paralleled results for the full cohort (data not shown). Statistical comparisons between serologic tests in the HIV-specific subsets could not be performed either because results were identical between tests or because all subjects were classified as positive or negative by one or both tests.
Simple and accurate HSV-2 serologic assays for African populations are needed if interventions to prevent HIV-1 transmission or delay HIV-1 disease progression are shown effective. Our comparison of HSV-2 serologic assays from samples of a community-based HSV-2 serosurveillance study in Kampala, Uganda indicate that as a single assay, Kalon had the best overall sensitivity and specificity . However, if Biokit is used to confirm Focus 1.1 positive results, the sensitivity and specificity of this combination was comparable to the Kalon assay. The addition of Biokit confirmation is analogous to current diagnostic testing for syphilis using a highly sensitivity screening test (e.g. VDRL or RPR for syphilis) followed by a highly specific confirmatory (i.e. treponemal-specific) assay. However, the use of Biokit confirmation of Focus positives does add to the total cost of the evaluation.
In lieu of confirmation with Biokit or another test, studies have suggested Focus assay performance may be improved in African populations by increasing the index value cutoff for a positive result [20–22, 29, 30]. Our findings corroborate that an optimized Focus index value cutoff of 2.2 was associated with higher specificity than the standard cutoff of 1.1 (92.4% versus 70.2%) with only modest reduction in sensitivity (96.4% versus 99.5%). Studies involving other African populations may help further define a more universal cutoff that could be applied to Focus testing in Africans. However, a practical consequence of an adjusted index cutoff will be an increase in the proportion of sera with HSV-2 testing results below the low positive cutoff, some of whom will be individuals in the process of seroconverting, thus requiring specific counseling messages and follow-up testing.
Notably, the study samples were a selected subset of the original Kampala community-based cohort with a bias toward including specimens with a Focus result in the “low positive” range. Historically, such samples have been an important source of discrepancy for different HSV-2 serologic assays and oversampling these samples provided increased power to discern differences in how the Kalon and Biokit assays performed on these. In order to ensure that our results appropriately reflected the original study population, we implemented statistical adjustments to account for this biased sample selection.
Although development of more robust assays will be the best approach for HSV-2 testing in Africa, our data support some practical options for using existing commercial HSV-2 serologic assays for HSV-2 diagnosis and management, and for development of HIV-1 prevention strategies in Africa: Kalon assay alone, a combination of Focus 1.1 and Biokit, and Focus 2.2 all demonstrate comparable sensitivity and specificity. However, each option is accompanied by a different tradeoff: Focus 1.1 had higher rates of false negative findings (specificity 70.9%) in HIV-1 infected persons; Focus plus Biokit would have higher testing cost; and Focus 2.2 leaves a higher proportion of subjects who have recently HSV-2 seroconverted in need of subsequent evaluations to clarify their HSV-2 status. Furthermore, the predictive value of any assay will need to be systematically assessed, particularly in the context of high HSV-2 prevalence such as would be encountered among HIV-infected populations. These are all important practical considerations that will impact on which assay is selected for HSV-2 testing in Uganda and elsewhere in Africa. Regional and population differences may require assessment prior to implementation of HSV-2 serologic screening programs.
In summary, recent and ongoing clinical trials are evaluating the role of HSV-2 suppressive and episodic therapy for public health and clinical benefits among HIV-1 infected persons. While our results indicate that existing commercial tests can be used, either by using the Kalon EIA with the manufacturer’s cutoff, by increasing the cut-off for the Focus EIA or by confirming positive assays with Biokit. Additional HSV-2 serologic assays with higher sensitivity and specificity, ideally in a point-of-care format, for African populations are needed so that as effective HSV-2 interventions are identified, such as better therapeutic drugs for HSV-2 and ultimately an HSV-2 vaccine, they can be implemented both for HSV-2 and HIV-1 prevention.
Funding for the community serosurveillance study in Kampala was supported by NIH Research Grant # D43 TW006672 funded by the Fogarty International Center. Western blot testing was supported by National Institutes of Health grant P01AI30731-17
Statement of Conflict of Interest:
The authors on this manuscript have no commercial or other conflicts of interest related to this research.
Prior meeting presentations:
The comparative serology data presented in this manuscript has not been presented in any other meetings or publications. Data from the “Seroprevalence and incidence of genital herpes in Uganda” study that generated the specimens tested in this study were presented at the 17th ISSTDR Meeting/10th IUSTI World Congress, Seattle, USA, July 2007.
All authors have reviewed and approved this manuscript