Background. The Step Study tested whether an adenovirus serotype 5 (Ad5)–vectored human immunodeficiency virus (HIV) vaccine could prevent HIV acquisition and/or reduce viral load set-point after infection. At the first interim analysis, nonefficacy criteria were met. Vaccinations were halted; participants were unblinded. In post hoc analyses, more HIV infections occurred in vaccinees vs placebo recipients in men who had Ad5-neutralizing antibodies and/or were uncircumcised. Follow-up was extended to assess relative risk of HIV acquisition in vaccinees vs placebo recipients over time.
Methods. We used Cox proportional hazard models for analyses of vaccine effect on HIV acquisition and vaccine effect modifiers, and nonparametric and semiparametric methods for analysis of constancy of relative risk over time.
Results. One hundred seventy-two of 1836 men were infected. The adjusted vaccinees vs placebo recipients hazard ratio (HR) for all follow-up time was 1.40 (95% confidence interval [CI], 1.03–1.92; P = .03). Vaccine effect differed by baseline Ad5 or circumcision status during first 18 months, but neither was significant for all follow-up time. The HR among uncircumcised and/or Ad5-seropositive men waned with time since vaccination. No significant vaccine-associated risk was seen among circumcised, Ad5-negative men (HR, 0.97; P = 1.0) over all follow-up time.
Conclusions. The vaccine-associated risk seen in interim analysis was confirmed but waned with time from vaccination.
Clinical Trials Registration. NCT00095576.
Recent advances in assay technology have led to major improvements in how HIV-1 neutralizing antibodies are measured. A luciferase reporter gene assay performed in TZM-bl (JC53bl-13) cells has been optimized and validated. Because this assay has been adopted by multiple laboratories worldwide, an external proficiency testing program was developed to ensure data equivalency across laboratories performing this neutralizing antibody assay for HIV/AIDS vaccine clinical trials.
The program was optimized by conducting three independent rounds of testing, with an increased level of stringency from the first to third round. Results from the participating domestic and international laboratories improved each round as factors that contributed to inter-assay variability were identified and minimized. Key contributors to increased agreement were experience among laboratories and standardization of reagents.
A statistical qualification rule was developed using a simulation procedure based on the three optimization rounds of testing, where a laboratory qualifies if at least 25 of the 30 ID50 values lie within the acceptance ranges. This ensures no more than a 20% risk that a participating laboratory fails to qualify when it should, as defined by the simulation procedure. Five experienced reference laboratories were identified and tested a series of standardized reagents to derive the acceptance ranges for pass–fail criteria. This Standardized Proficiency Testing Program is the first available for the evaluation and documentation of assay equivalency for laboratories performing HIV-1 neutralizing antibody assays and may provide guidance for the development of future proficiency testing programs for other assay platforms.
Neutralizing; Antibody; Assay; Proficiency; HIV; TZM-bl
We address the problem of establishing two-sided equivalence using paired-sample analysis of two treatments or two laboratory tests with a binary endpoint. Through real data examples and monte carlo simulations, we compare three commonly used testing parameters, namely, the difference of response probabilities, the ratio of response probabilities, and the ratio of discordant probabilities based on score test statistics for constructing equivalence hypothesis tests of paired binary data. We provide suggestions on the choice of these three testing parameters and proper equivalence margins in hypothesis formulation of equivalence testing. In addition, we describe the implementation of a group sequential design in the context of equivalence testing with early stopping to reject, as well as to declare equivalence.
binary data; double one-sided testing; equivalence boundary; interim analysis; sample sizes
With the development of novel assay technologies, biomedical experiments and analyses have gone through substantial evolution. Today, a typical experiment can simultaneously measure hundreds to thousands of individual features (e.g. genes) in dozens of biological conditions, resulting in gigabytes of data that need to be processed and analyzed. Because of the multiple steps involved in the data generation and analysis and the lack of details provided, it can be difficult for independent researchers to try to reproduce a published study. With the recent outrage following the halt of a cancer clinical trial due to the lack of reproducibility of the published study, researchers are now facing heavy pressure to ensure that their results are reproducible. Despite the global demand, too many published studies remain non-reproducible mainly due to the lack of availability of experimental protocol, data and/or computer code. Scientific discovery is an iterative process, where a published study generates new knowledge and data, resulting in new follow-up studies or clinical trials based on these results. As such, it is important for the results of a study to be quickly confirmed or discarded to avoid wasting time and money on novel projects. The availability of high-quality, reproducible data will also lead to more powerful analyses (or meta-analyses) where multiple data sets are combined to generate new knowledge. In this article, we review some of the recent developments regarding biomedical reproducibility and comparability and discuss some of the areas where the overall field could be improved.
Analysis pipeline; accuracy; open science; precision; protocol; standardization
Background. To investigate the potential immunostimulatory effect of interleukin (IL) 2 as a human immunodeficiency virus type 1 (HIV-1) vaccine adjuvant, we conducted a study of a plasmid coding for a fusion protein of IL-2 and immunoglobulin (IL-2/Ig).
Methods. This phase I trial evaluated an HIV-1 DNA vaccine with the plasmid cytokine adjuvant (IL-2/Ig) in 70 HIV-negative adults. Subjects received placebo (group C), adjuvant alone (group A), vaccine alone (group D), increasing doses of adjuvant concurrent with vaccine (groups T1–T4), or adjuvant given 2 days after vaccine (group T5).
Results. No significant differences in adverse events were observed between treatment groups. Cellular immune responses to envelope protein EnvA peptides were detected by interferon (IFN) γ and IL-2 enzyme-linked immunospot (ELISPOT) assays in 50% and 40% of subjects, respectively, in T4, and in 100% and 80% in T5. The median responses for groups T4 and T5, respectively, were 90 and 193 spot-forming cells (SFCs)/106 peripheral blood mononuclear cells (P = .004; T4 vs T5) for the IL-2 ELISPOT assay and 103 and 380 SFCs/106 PBMCs (P = .003; T4 vs T5) for the IFN-γ ELISPOT assay. A trend to more durable cellular immune responses in T5 was observed at 1 year (T5 vs T4/D; P = .07). Higher anti-Env antibody responses were detected with T5 than with T4.
Conclusions. Plasmid IL-2/Ig significantly increased immune responses when administered 2 days after the DNA vaccine, compared with simultaneous administration. These observations have important implications for the development of cytokine augmentation strategies.
Clinical Trials Registration. NCT00069030.
Most T cell-based HIV-1 vaccine candidates induce responses of limited breadth for reasons that are unclear. We evaluated vaccine-induced T-cell responses in individuals receiving an HIV-1 recombinant adenoviral vaccine. Certain HLA alleles (B27, B57, B35 and B14) are preferentially utilized to mount HIV-specific responses, whereas other alleles (A02 and B07) are rarely utilized (p<0.001). This preference appears due to four factors individually or in combination: higher affinity of specific peptides to specific HLA alleles; higher avidity of TCR; HLA and peptide interaction; and/or higher surface expression of certain HLA. Thus, HLA immunodominance plays a substantial role in vaccine-induced T-cell responses.
HIV vaccine; immunodominance; preferential HLA usage
In the RV144 trial, the estimated efficacy of a vaccine regimen against human immunodeficiency virus type 1 (HIV-1) was 31.2%. We performed a case–control analysis to identify antibody and cellular immune correlates of infection risk.
In pilot studies conducted with RV144 blood samples, 17 antibody or cellular assays met prespecified criteria, of which 6 were chosen for primary analysis to determine the roles of T-cell, IgG antibody, and IgA antibody responses in the modulation of infection risk. Assays were performed on samples from 41 vaccinees who became infected and 205 uninfected vaccinees, obtained 2 weeks after final immunization, to evaluate whether immune-response variables predicted HIV-1 infection through 42 months of follow-up.
Of six primary variables, two correlated significantly with infection risk: the binding of IgG antibodies to variable regions 1 and 2 (V1V2) of HIV-1 envelope proteins (Env) correlated inversely with the rate of HIV-1 infection (estimated odds ratio, 0.57 per 1-SD increase; P = 0.02; q = 0.08), and the binding of plasma IgA antibodies to Env correlated directly with the rate of infection (estimated odds ratio, 1.54 per 1-SD increase; P = 0.03; q = 0.08). Neither low levels of V1V2 antibodies nor high levels of Env-specific IgA antibodies were associated with higher rates of infection than were found in the placebo group. Secondary analyses suggested that Env-specific IgA antibodies may mitigate the effects of potentially protective antibodies.
This immune-correlates study generated the hypotheses that V1V2 antibodies may have contributed to protection against HIV-1 infection, whereas high levels of Env-specific IgA antibodies may have mitigated the effects of protective antibodies. Vaccines that are designed to induce higher levels of V1V2 antibodies and lower levels of Env-specific IgA antibodies than are induced by the RV144 vaccine may have improved efficacy against HIV-1 infection.
Extensive observational data suggest that HSV-2 infection may
facilitate HIV acquisition, increase HIV viral load, and accelerate HIV
progression and onward transmission. To explore these relationships, we
examined the impact of pre-existing HSV-2 infection in an international HIV
We analyzed the associations between prevalent HSV-2 infection and
HIV-1 acquisition and progression among 1836 men who have sex with men
(MSM). We used Cox proportional hazards regression models to estimate the
association between HSV-2 infection and both HIV acquisition and ART
initiation, and linear regression to explore the effect of HSV-2 on pre-ART
HSV-2 infection increased risk of HIV-1 acquisition among all
volunteers (adjusted hazard ratio 2.2; 95% CI, 1.4 to 3.5).
Adjusting for demographic variables, circumcision, Ad5 titer and significant
risk behaviors, the risk of HIV acquisition among HSV-2 infected placebo
recipients was three fold higher than HSV-2 seronegatives (hazard ratio 3.3;
95% CI, 1.6 to 6.9). Past HSV-2 infection was associated with a 0.2
log10 copies/ml higher adjusted mean set point viral load
(95% CI, 0.3 lower to 0.6 higher). HSV-2 infection was not
associated with time to ART initiation.
Among MSM in an HIV-1 vaccine trial, pre-existing HSV-2 infection was
a major risk factor for HIV acquisition. Past HSV-2 did not significantly
increase HIV viral load or early disease progression. HSV-2 seropositive
persons will likely prove more difficult than HSV-2 seronegative persons to
protect against HIV infection using vaccines or other prevention
Herpes Simplex Virus Type II; HIV incidence
Many candidate HIV vaccines are designed to primarily elicit T-cell responses. Although repeated immunization with the same vaccine boosts antibody responses, the benefit for T-cell responses is ill-defined. We compared two immunization regimens that include the same recombinant adenoviral serotype 5 (rAd5) boost. Repeated homologous rAd5 immunization fails to increase T-cell responses, but increases gp140 antibody responses ten-fold. DNA prime, as compared with rAd5 prime, directs long-term memory CD8+ T cells toward a terminally differentiated effector memory phenotype with cytotoxic potential. Based on the kinetics of activated cells measured directly ex vivo, the DNA vaccination primes for both CD4+ and CD8+ T cells, despite the lack of detection of the latter until after the boost. These results suggest that heterologous prime-boost combinations have distinct immunological advantages over homologous prime-boosts, and suggest that the effect of DNA on subsequent boosting may not be easily detectable directly after the DNA vaccination.
Background. A key missing element in the development of a successful human immunodeficiency virus (HIV) vaccine is an immunogen that can generate broadly cross-neutralizing antibodies against primary isolates of the virus.
Methods. This phase 1 clinical trial employed a DNA prime and subunit envelope protein boost in an attempt to generate cellular and humoral immune responses that might be desirable in a protective HIV vaccine. Priming was performed via intramuscular injection with gag and env DNA adsorbed to polylactide coglycolide microspheres, followed by boosting with a recombinant trimeric envelope (Env) glycoprotein delivered in MF59 adjuvant.
Results. The DNA prime and protein boost were generally safe and well-tolerated. Env-specific CD4+ cellular responses were generated that were predominantly detected after Env protein boosting. Neutralizing antibody responses against the homologous SF162 viral isolate were remarkably strong and were present in the majority of vaccine recipients, including a strong response against CD4-induced epitopes on gp120. Despite the promising potency of this vaccine approach, neutralization breadth against heterologous tier 2 strains of HIV-1 was minimal.
Conclusions. Potent neutralization against neutralization-sensitive strains of HIV is achievable in humans through a DNA prime, recombinant oligomeric Env protein boost regimen. Eliciting substantial breadth of neutralization remains an elusive goal.
Clinical Trials Registration. NCT00073216.
Recombinant viruses hold promise as vectors for vaccines to prevent infectious diseases with significant global health impacts. One of their major limitations is that preexisting anti-vector neutralizing antibodies can reduce T cell responses to the insert antigens; however, the impact of vector-specific cellular immunity on subsequent insert-specific T cell responses has not been assessed in humans. Here, we have identified and compared adenovirus-specific and HIV-specific T cell responses in subjects participating in two HIV-1 vaccine trials using a vaccine vectored by adenovirus serotype 5 (Ad5). Higher frequencies of pre-immunization adenovirus-specific CD4+ T cells were associated with substantially decreased magnitude of HIV-specific CD4+ T cell responses and decreased breadth of HIV-specific CD8+ T cell responses in vaccine recipients, independent of type-specific preexisting Ad5-specific neutralizing antibody titers. Further, epitopes recognized by adenovirus-specific T cells were commonly conserved across many adenovirus serotypes, suggesting that cross-reactivity of preexisting adenovirus-specific T cells can extend to adenovirus vectors derived from rare serotypes. These findings provide what we believe to be a new understanding of how preexisting viral immunity may impact the efficacy of vaccines under current evaluation for prevention of HIV, tuberculosis, and malaria.
The results of the recent Step Study highlight a need to clarify the effects of pre-existing natural immunity to a vaccine vector on vaccine-induced T-cell responses. To investigate this interaction, we examined the relationship between pre-existing Ad5 immunity and T-cell cytokine response profiles in healthy, HIV-uninfected recipients of MRKAd5 HIV-1 gag vaccine (HVTN 050, ClinicalTrials.gov #NCT00849732). Participants were grouped by baseline Ad5 neutralizing antibody titer as either Ad5-seronegative (titer ≤18; n = 36) or Ad5-seropositive (titer >200; n = 34). Samples from vaccine recipients were analyzed for immune responses to either HIV-1 Gag peptide pools or Ad5 empty vector using an ex vivo assay that measures thirty cytokines in the absence of long-term culture. The overall profiles of cytokine responses to Gag and Ad5 had similar combinations of induced Th1- and Th2-type cytokines, including IFN-γ, IL-2, TNF-α, IP-10, IL-13, and IL-10, although the Ad5-specific responses were uniformly higher than the Gag-specific responses (p<0.0001 for 9 out of 11 significantly expressed analytes). At the peak response time point, PBMC from Ad5-seronegative vaccinees secreted significantly more IP-10 in response to Gag (p = 0.008), and significantly more IP-10 (p = 0.0009), IL-2 (p = 0.006) and IL-10 (p = 0.05) in response to Ad5 empty vector than PBMC from Ad5-seropositive vaccinees. Additionally, similar responses to the Ad5 vector prior to vaccination were observed in almost all subjects, regardless of Ad5 neutralizing antibody status, and the levels of secreted IFN-γ, IL-10, IL-1Ra and GM-CSF were blunted following vaccination. The cytokine response profile of Gag-specific T cells mirrored the Ad5-specific response present in all subjects before vaccination, and included a number of Th1- and Th2-associated cytokines not routinely assessed in current vaccine trials, such as IP-10, IL-10, IL-13, and GM-CSF. Together, these results suggest that vector-specific humoral responses may reduce vaccine-induced T-cell responses by previously undetected mechanisms.
The Comprehensive T Cell Vaccine Immune Monitoring Consortium (CTC-VIMC) was created to provide standardized immunogenicity monitoring services for HIV vaccine trials. The ex vivo interferon-gamma (IFN-γ) ELISpot is used extensively as a primary immunogenicity assay to assess T cell-based vaccine candidates in trials for infectious diseases and cancer. Two independent, GCLP-accredited central laboratories of CTC-VIMC routinely use their own standard operating procedures (SOPs) for ELISpot within two major networks of HIV vaccine trials. Studies are imperatively needed to assess the comparability of ELISpot measurements across laboratories to benefit optimal advancement of vaccine candidates.
We describe an equivalence study of the two independently qualified IFN-g ELISpot SOPs. The study design, data collection and subsequent analysis were managed by independent statisticians to avoid subjectivity. The equivalence of both response rates and positivity calls to a given stimulus was assessed based on pre-specified acceptance criteria derived from a separate pilot study.
Detection of positive responses was found to be equivalent between both laboratories. The 95% C.I. on the difference in response rates, for CMV (−1.5%, 1.5%) and CEF (−0.4%, 7.8%) responses, were both contained in the pre-specified equivalence margin of interval [−15%, 15%]. The lower bound of the 95% C.I. on the proportion of concordant positivity calls for CMV (97.2%) and CEF (89.5%) were both greater than the pre-specified margin of 70%. A third CTC-VIMC central laboratory already using one of the two SOPs also showed comparability when tested in a smaller sub-study.
The described study procedure provides a prototypical example for the comparison of bioanalytical methods in HIV vaccine and other disease fields. This study also provides valuable and unprecedented information for future vaccine candidate evaluations on the comparison and pooling of ELISpot results generated by the CTC-VIMC central core laboratories.
Tenofovir gel has entered into clinical trials for use as a topical microbicide to prevent HIV-1 infection but has no published data regarding pre-clinical testing using in vitro and ex vivo models. To validate our findings with on-going clinical trial results, we evaluated topical tenofovir gel for safety and efficacy. We also modeled systemic application of tenofovir for efficacy.
Methods and Findings
Formulation assessment of tenofovir gel included osmolality, viscosity, in vitro release, and permeability testing. Safety was evaluated by measuring the effect on the viability of vaginal flora, PBMCs, epithelial cells, and ectocervical and colorectal explant tissues. For efficacy testing, PBMCs were cultured with tenofovir or vehicle control gels and HIV-1 representing subtypes A, B, and C. Additionally, polarized ectocervical and colorectal explant cultures were treated apically with either gel. Tenofovir was added basolaterally to simulate systemic application. All tissues were challenged with HIV-1 applied apically. Infection was assessed by measuring p24 by ELISA on collected supernatants and immunohistochemistry for ectocervical explants. Formulation testing showed the tenofovir and vehicle control gels were >10 times isosmolar. Permeability through ectocervical tissue was variable but in all cases the receptor compartment drug concentration reached levels that inhibit HIV-1 infection in vitro. The gels were non-toxic toward vaginal flora, PBMCs, or epithelial cells. A transient reduction in epithelial monolayer integrity and epithelial fracture for ectocervical and colorectal explants was noted and likely due to the hyperosmolar nature of the formulation. Tenofovir gel prevented HIV-1 infection of PBMCs regardless of HIV-1 subtype. Topical and systemic tenofovir were effective at preventing HIV-1 infection of explant cultures.
These studies provide a mechanism for pre-clinical prediction of safety and efficacy of formulated microbicides. Tenofovir was effective against HIV-1 infection in our algorithm. These data support the use of tenofovir for pre-exposure prophylaxis.
Both the magnitude and breadth of neutralization against multiple strains of virus are main endpoints for comparing antibody-based HIV-1 vaccine candidates in Phase I and II trials, and are key markers to be evaluated in vaccine efficacy trials as immune correlates of protection against HIV-1 infection. More generally, consideration of both magnitude and breadth is encountered when there is interest in comparing quantitative multivariate response data between groups. In this paper, we discuss two approaches to simultaneously evaluating the magnitude and breadth of a multivariate response. We suggest methods for the summarization and group comparison of multivariate response data that are subject to left and/or right censoring. Applications to data from a phase III clinical trial (Vax004) are discussed. Simulation-based sample size calculations and power analyses of the described methods also are presented.
Censored data; Group comparison; Immunological data; Multivariate data; Sample size
Intracellular cytokine staining (ICS) by multiparameter flow cytometry is one of the primary methods for determining T cell immunogenicity in HIV-1 clinical vaccine trials. Data analysis requires considerable expertise and time. The amount of data is quickly increasing as more and larger trials are performed, and thus there is a critical need for high throughput methods of data analysis.
A web based flow cytometric analysis system, LabKey Flow, was developed for analyses of data from standardized ICS assays. A gating template was created manually in commercially-available flow cytometric analysis software. Using this template, the system automatically compensated and analyzed all data sets. Quality control queries were designed to identify potentially incorrect sample collections.
Comparison of the semi-automated analysis performed by LabKey Flow and the manual analysis performed using FlowJo software demonstrated excellent concordance (concordance correlation coefficient >0.990). Manual inspection of the analyses performed by LabKey Flow for 8-color ICS data files from several clinical vaccine trials indicates that template gates can appropriately be used for most data sets.
The semi-automated LabKey Flow analysis system can analyze accurately large ICS data files. Routine use of the system does not require specialized expertise. This high-throughput analysis will provide great utility for rapid evaluation of complex multiparameter flow cytometric measurements collected from large clinical trials.
Flow Cytometry; Intracellular Cytokine Staining; HIV-1; vaccine; T cell; immunogenicity; data analysis; automation
Candidate HIV-1 vaccines currently being evaluated in clinical trials are designed to elicit HIV-1-specific cellular immunity. Intracellular cytokine staining (ICS) assays allow sensitive, quantitative ex vivo assessments of antigen-specific T cells including immunophenotyping of responding cells and measurement of multiple effector functions. Additionally, the use of banked cryopreserved PBMC samples makes this assay attractive in the setting of large efficacy trials where it is less feasible to perform immunoassays on freshly isolated samples. Here we describe extensive studies to optimize and quantitatively validate the 8-color ICS assay for use in clinical trials of candidate vaccines, which includes measurement of viable IFN-γ, IL-2, TNF-α and IL-4 secreting CD4+ and CD8+ T cells. We show that omission of viability dye staining results in an over-estimate of the true antigen-specific T cell response by up to two-fold. After optimization, the 8-color assay was validated for specificity, precision, linearity, limit of quantitation and robustness. The assay has a lower quantitation limit, generally below 0.04%, depending on the cytokine subset. Additionally, with appropriate gating, the 8-color assay gives comparable cytokine-positive responses to those observed with the conventional 4-color assay. In conclusion, we provide the first description of a quantitatively validated ICS assay, which permits quantitative and qualitative evaluation of vaccine-induced immunogenicity and analysis of immune correlates of protection.
Flow cytometry; intracellular cytokine staining; vaccine; immunogenicity; assay validation
Progressive immune dysfunction and AIDS develop in most cases of human immunodeficiency virus type 1 (HIV-1) infection but in only 25 to 30% of persons with HIV-2 infection. However, the natural history and immunologic responses of individuals with dual HIV-1 and HIV-2 infection are largely undefined. Based on our previous findings, we hypothesized that among patients with dual infection the control of HIV-1 is associated with the ability to respond to HIV-2 Gag epitopes and to maintain HIV-specific CD4+ T-cell responses. To test this, we compared the HIV-specific ex vivo IFN-γ enzyme-linked immunospot (ELISPOT) assay responses of 19 dually infected individuals to those of persons infected with HIV-1 or HIV-2 only. Further, we assessed the functional profile of HIV Gag-specific CD4+ and CD8+ T cells from nine HIV dually infected patients by using a multicolor intracellular cytokine staining assay. As determined by ELISPOT assay, the magnitude and frequency of IFN-γ-secreting T-cell responses to gene products of HIV-1 were higher than those to gene products of HIV-2 (2.64 versus 1.53 log10 IFN-γ spot-forming cells/106 cells [90% versus 63%, respectively].) Further, HIV-1 Env-, Gag-, and Nef- and HIV-2 Gag-specific responses were common; HIV-2 Nef-specific responses were rare. HIV-specific CD4+ T helper responses were detected in nine of nine dually infected subjects, with the majority of these T cells producing gamma interferon (IFN-γ) and tumor necrosis factor alpha (TNF-α) and, to a lesser extent, interleukin-2. The HIV-1 plasma viral load was inversely correlated with HIV-2 Gag-specific IFN-γ-/TNF-α-secreting CD4+ and HIV-2 Gag-specific IFN-γ-secreting CD8+ T cells. In conclusion, the T-cell memory responses associated with containment of single HIV-1 and HIV-2 infection play a similar significant role in the immune control of dual HIV-1 and HIV-2 infection.