Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Curr Opin Virol. Author manuscript; available in PMC 2017 April 1.
Published in final edited form as:
PMCID: PMC4902743

Selection of HIV Vaccine Candidates for Concurrent Testing in an Efficacy Trial


Phase IIb or III HIV-1 vaccine efficacy trials are generally large and operationally challenging. To mitigate this challenge, the HIV Vaccine Trials Network is designing a Phase IIb efficacy trial accommodating the evaluation of multiple vaccine regimens concurrently. As this efficacy trial would evaluate a limited number of vaccine regimens, there is a need to develop a framework for optimizing the strategic selection of regimens from the large number of vaccine candidates tested in Phase I/IIa trials. In this paper we describe the approaches for the selection process, including the choice of immune response endpoints and the statistical criteria and algorithms. We illustrate the selection approaches using data from HIV-1 vaccine trials.


The World Health Organization [1], and the Joint United Nations Programme on HIV/AIDS [2], estimate that there were 36.9 million people living with HIV globally by the end of 2014, with Sub-Saharan Africa accounting for 70% of the global HIV burden. Development of a preventive HIV vaccine remains a global health priority. Unfortunately, despite decades of research, only a single study, RV144, has demonstrated modest protection against HIV. This study was a phase III randomized controlled study conducted in Thailand. The vaccine regimen (two doses of prime with ALVAC-HIV (vCP1521), followed by two boosts with ALVAC-HIV + AIDSVAX clades B/E gp120 protein) demonstrated 31.2% efficacy compared to placebo (p = 0.04) at 3.5 years [3] and 60.5% efficacy through 12 months [4]. These results have reinvigorated the scientific community by suggesting that developing a preventive HIV vaccine may be possible.

To better understand how the RV144 vaccine regimen reduced the risk of HIV infection, a large consortium of independent laboratories worked together systematically to perform a case control study within the RV144 trial to identify correlates of risk (CoR) for HIV infection [5], i.e. vaccine-induced immune response biomarkers that are associated with subsequent HIV infection [6, 7]. Two CoRs were identified: immunoglobulin G (IgG) Antibody that binds to a scaffolded gp70 V1V2 recombinant protein (inversely correlated with risk); and plasma Env-specific binding immunoglobulin A (IgA) (directly correlated with risk). Four additional variables correlated inversely with infection risk when the level of IgA binding was low. Recently, several studies have further enhanced our understanding of the efficacy seen in RV144 [8, 9, 10, 11, 12, 13, 14, 15] and the potential relevance of Env V1V2-specific IgG3 [16, 17]. These studies lay the groundwork for immunogenicity analyses in several of the ongoing and future HIV vaccine trials.

Since its inception in the late nineties, the HIV Vaccine Trials Network (HVTN) has been conducting multiple clinical trials with a large number of candidate HIV vaccine regimens in different regions of the world. The vast majority of these studies arePhase I/IIa safety and immunogenicity studies, with only a few Phase IIb trials evaluating the impact of vaccine candidates on HIV-1 infection [18, 19, 20]. Phase IIb efficacy trials are generally large and operationally challenging. To mitigate this challenge, the HVTN is considering the inclusion of several vaccine regimens concurrently in one Phase IIb efficacy study, to augment study design and operational efficiencies [21, 22]. As this efficacy trial would only allow evaluation of a limited number of candidate vaccine regimens, we need a framework to select the most promising vaccine regimens from the reagents tested in Phase I/IIa trials. We describe approaches for the selection process and illustrate our rationale using data from completed HIV-1 vaccine trials.


A number of phase I/IIa studies testing multiple vaccine regimens comprising combinations of non-protein components (viral vectors or DNA), Env proteins and adjuvants, and administered at different schedules (3 injections at 0, 1 and 6 months or 4 injections at 0, 1, 3 and 6 months) will provide the data for selecting vaccine regimens for efficacy testing.

Three-Step Down-Selection Scheme

We plan a three-step scheme for down-selection (Figure 1). In Step 1, all regimens entering down-selection are screened based on safety and peak immunogenicity from a core set of immune assays measured on month 6.5 (two weeks after the last prime-boost vaccination) samples. In Steps 2 and 3, regimens passing Step 1 will be evaluated based on immunogenicity data from a full set of assays measured on month 6.5 samples, and a subset of months 3.5 (two weeks after the month 3 vaccination) and month 12 (6 months after the last vaccination) samples. The latter two time points are included for differentiating onset of immune responses among vaccine recipients and for assessing the durability of induced immune responses [23]. Step 2 involves a comparison of each putative regimen with the reference ALVAC-gp120 C/C in MF59 regimen, a similar regimen as the one used in RV144 but adapted for the region of southern Africa (HIV clade C based). ALVAC-gp120 C/C in MF59 is currently being tested in HVTN 100, a phase I trial in South Africa that may lead to a subsequent pivotal efficacy trial. Candidate regimens that are not superior to ALVAC-gp120 C/C for at least one immunological endpoint will be filtered out. Step 3 conducts head-to-head comparisons of the remaining regimens for final down-selection.

Figure 1
Down-selection scheme

All steps rely on the identification of CoRs and possible immune correlates of protection (CoP). The latter is an immune response biomarker that predicts vaccine efficacy [6, 7]. Although several potential CoRs were identified in RV144, no CoP has been validated in the HIV-1 vaccine field [5, 15]. One approach to assessing a potential CoP requires first demonstrating overall VE > 0, augmenting the efficacy trial design, and making additional assumptions in order to estimate the immune responses of placebo participants if they had received vaccine [24, 25, 26]. In order to maximize the possibility that selected vaccine regimens would confer protection in a future efficacy trial, the down-selection will incorporate existing knowledge on HIV-1 vaccines as well as emerging evidence and knowledge of mechanistic CoPs for other licensed vaccines.

Take/Potency Criteria for Step 1

An immunogenicity criterion called take/potency is considered in Step 1, which requires response rates above pre-specified thresholds for designated core immunological endpoints considered essential for a vaccine regimen to potentially confer adequate protection based on current knowledge. Specifically, the majority of vaccine recipients must generate IgG binding antibodies to the vaccine gp120s and a minimum frequency of vaccine recipients must generate two of three types of responses: V2 antibodies, neutralizing antibodies (NAb), and CD4+ or CD8+ T cell responses.

Down-selection based on Additional Immunological Endpoints for Steps 2 and 3

A potentially large number of immunological endpoints are considered in Steps 2 and 3, including all immune classes that are part of a putative CoP (Figure 2a). For each immune class, one or more endpoint scores will be used to summarize a vaccine recipient’s immune response in that class, which includes either a continuous score indicating magnitude or a binary score indicating positive/negative response. Down-selection will also be performed using a subset of immunological endpoints that were demonstrated to be statistically significant CoRs in RV144 (Figure 2b).

Figure 2Figure 2
Lists of immune classes for the second step of down-selection.

Head-to-Head Comparison between Regimens in Step 3

Selection Criteria

To down-select vaccine regimens based on comparisons of their multivariate immune response profiles in Step 3, we developed two criteria (Figure 3). The first is “superiority,” where selected vaccine regimens should be immunologically superior to un-selected regimens in a statistically defined way. Assuming a larger immune endpoint score is associated with a presumed better protective effect of vaccine, we define superiority for each endpoint score as a larger mean and consider a regimen to be superior to another with respect to its immune profile if it is superior with respect to at least one endpoint score and is not inferior with respect to any endpoint score. The second criterion is “non-redundancy.” The protective mechanisms of HIV-1 infection via the generation of different immune responses are not yet well understood. It is desirable to select vaccine regimens with non-redundant immune profiles [27], so that diverse mechanisms of vaccine protective effects can be investigated in the efficacy trial. We define two regimens A and B to be non-redundant if A is superior to B with respect to at least one endpoint score and B is superior to A with respect to at least one endpoint score.

Figure 3
Demonstration of the superiority and non-redundancy criteria. (a) Suppose six regimens with mean immune endpoint scores enter the down-selection. The pairwise relationship is shown in (b), where → indicates that one regimen is superior to the ...

Selection Algorithms

Based on these criteria, we developed two statistical algorithms for down-selection that integrates hypothesis testing, ranking, and clustering (Y Huang et al., unpublished). The first, the “ranking, filtering, and selection” (RFS) algorithm, 1) ranks all regimens according to an overall summary score that weights each endpoint score according to its putative predictive clinical importance to vaccine efficacy, 2) selects the top-ranked regimen, and 3) sequentially evaluates regimens ranked next, selecting regimens non-redundant with the existing set and excluding regimens inferior to the newly selected one, based on hypothesis tests of each individual endpoint score between regimens. To minimize errors due to multiple testing, we implement multiplicity correction to individual tests such that the probability of violating the non-redundancy criterion during down-selection can be controlled at a pre-specified level.

The second algorithm applies a “clustering and ranking” (CR) step before the RFS, which groups the candidate vaccine regimens into different clusters based on similarities in their immune response profiles and selects from each cluster the top-ranked regimen based on the aforementioned summary score to enter the RFS. We name this algorithm CR+RFS.

Data Example

As an example, we applied the down-selection algorithms to evaluate five regimens studied previously in HIV-1 vaccine trials: the RV144 vaccine regimen (RV144.T) and four regimens studied in the phase 1 trial HVTN096, including NYVAC prime plus NYVAC + AIDSVAX B/E boosts (096.T1), NYVAC + AIDSVAX B/E prime plus NYVAC + AIDSVAX B/E boosts (096.T2), DNA prime plus NYVAC + AIDSVAX B/E boosts (096.T3), and DNA + AIDSVAX B/E prime plus NYVAC + AIDSVAX B/E boosts (096.T4). Peak immune assay data at month 6.5 were available for 205, 19, 18, 17, and 19 HIV-1 uninfected vaccinees, respectively, for eight immunological endpoints common across trials including IgG binding antibody responses to six different gp120 antigens, mean NAb responses to six HIV-1 isolates and CD4+ T-cell response. We generated a dataset that included 43 individuals per regimen by sampling with replacement from individuals with complete endpoint scores within each regimen.

For the demonstration we equally weighted IgG, CD4+, and NAb and subdivided the weight for IgG equally between the six different antigens. The ranking of the five regimens based on the weighted average score from highest to lowest was 096.T3, 096.T1, 096.T4, 096.T2, and RV144.T (Figure 4a). We use a principal component biplot to graphically display the structure of the data (Figure 4b). It showed that the IgG and NAb scores were correlated and were along the first principal component direction capturing the maximum variation in responses, whereas the CD4+ score captured additional variation more in the second principal component direction. Hierarchical clustering with weighted Manhattan distance [28] grouped the five vaccine regimens into four clusters: {RV144.T}, {096.T1, 096.T3}, {096.T2}, {096.T4} (Figure 4c–d). Both RFS and CR+RFS algorithms selected 096.T3 only; the other regimens were either inferior or redundant and not superior (Figure 4d).

Figure 4Figure 4
Results of the data example. (a) Assay-specific mean immune endpoint scores for five HIV-1 vaccine regimens. The eight endpoint scores are: IgG binding antibody responses to six different gp120 antigens measured using the binding antibody multiplex assay ...


The development of the down-selection plan is a multi-disciplinary collaboration demanding sophisticated statistical and scientific considerations. The framework is a new advancement for the field of HIV vaccine research: the selection of more than one regimen would lead to the first multi-regimen HIV vaccine efficacy trial.

The down-selection framework is broadly applicable. It can accommodate different assumptions about immune response profiles that are consistent with protection using different ranking strategies, e.g., by favoring regimens with best immune response on average, or best immune responses in a few classes, etc. This flexibility is essential given the lack of full knowledge about the nature of immune responses needed for protection. The choice of weights of immune response endpoints affects selection outcome, the finalization of which requires continuing discussions among researchers and updates to accommodate emerging research in the field. Lastly, the framework was developed with focus on candidate vaccines in a relatively homogenous class of regimens (pox-protein), and additional research would be needed to optimize the framework for regimens generating totally novel responses (e.g., broadly neutralizing Tier 2 antibodies).

Performance of the down-selection algorithms depends critically on the sample sizes and the differences between regimens. Simulation studies of the phase I/IIa trials demonstrated reasonable selection performance if regimens have immunological differences consistent with a 20% difference in vaccine efficacy, based on CoR and CoP analyses of RV144 (Supplementary Figure 1).


  • Selection of more than 1 regimen would lead to the first multi-regimen HIV vaccine efficacy trial
  • Choices of immune correlates for down-selection are both knowledge-based and comprehensive
  • The approach has good performance to differentiate medium to large immunological differences

Supplementary Material



The authors thank the participants, investigators, and sponsors of the HVTN096 and RV144 trials.

The authors thank the HVTN lab program for generating the immune response data for the HVTN096 trial and Dr. Gepi Pantaleo for chairing the HVTN096 trial. This research is supported by NIH NIAID award UM1AI068635.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of interest statement:

Carlos DiazGranados and Sanjay Phogat are fulltime employees of Sanofi Pasteur.


1. WHO HIV/AIDS Fact sheet. [Updated July 2015]; Available at:
2. UNAIDS. MDG 6: 15 years, 15 lessons of hope from the AIDS response fact sheet. Available at
3. Rerks-Ngarm S, Pitisuttithum P, Nitayaphan S, Kaewkungwal J, Chiu J, Paris R, Premsri N, Namwat C, de Souza M, Adams E, et al. Vaccination with ALVAC and AIDSVAX to prevent HIV-1 infection in Thailand. N Engl J Med. 2009;361(23):2209–2220. [PubMed]
4. Robb ML, Rerks-Ngarm S, Nitayaphan S, Pitisuttithum P, Kaewkungwal J, Kunasol P, Khamboonruang C, Thongcharoen P, Morgan P, Benenson M, et al. Risk behaviour and time as covariates for efficacy of the HIV vaccine regimen ALVAC-HIV (vCP1521) and AIDSVAX B/E: a post-hoc analysis of the Thai phase 3 efficacy trial RV144. The Lancet Infectious Diseases. 2012;12(7):531–537. [PMC free article] [PubMed]
5. Haynes BF, Gilbert PB, McElrath MJ, Zolla-Pazner S, Tomaras GD, Alam SM, Evans DT, Montefiori DC, Karnasuta C, Sutthent R, et al. Immune-correlates analysis of an HIV-1 vaccine efficacy trial. N Engl J Med. 2012;366(14):1275–1286. [PMC free article] [PubMed]
6. Qin L, Gilbert PB, Corey L, McElrath MJ, Self SG. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis. 2007;196(9):1304–1312. [PubMed]
7. Plotkin SA, Gilbert PB. Nomenclature for immune correlates of protection after vaccination. Clin Infect Dis. 2012;54(11):1615–1617. [PMC free article] [PubMed]
8. Rolland M, Edlefsen PT, Larsen BB, Tovanabutra S, Sanders-Buell E, Hertz T, deCamp AC, Carrico C, Menis S, Magaret CA, et al. Increased HIV-1 vaccine efficacy against viruses with genetic signatures in Env V2. Nature. 2012;490(7420):417–420. [PMC free article] [PubMed]
9. Liao HX, Bonsignori M, Alam SM, McLellan JS, Tomaras GD, Moody MA, Kozink DM, Hwang KK, Chen X, Tsao CY, et al. Vaccine induction of antibodies against a structurally heterogeneous site of immune pressure within HIV-1 envelope protein variable regions 1 and 2. Immunity. 2013;38(1):176–186. [PMC free article] [PubMed]
10. Gottardo R, Bailer RT, Korber BT, Gnanakaran S, Phillips J, Shen X, Tomaras GD, Turk E, Imholte G, Eckler L, et al. Plasma IgG to linear epitopes in the V2 and V3 regions of HIV-1 gp120 correlate with a reduced risk of infection in the RV144 vaccine efficacy trial. PLoS One. 2013;8(9):e75665. [PMC free article] [PubMed]
11. Tomaras GD, Ferrari G, Shen X, Alam SM, Liao HX, Pollara J, Bonsignori M, Moody MA, Fong Y, Chen X. Vaccine-induced plasma IgA specific for the C1 region of the HIV-1 envelope blocks binding and effector function of IgG. Proceedings of the National Academy of Sciences. 2013;110(22):9019–9024. [PubMed] (*) This paper investigated the hypothesis that IgA could attenuate the protective effect of IgG responses through competition for the same Env binding sites. It showed that Env-specific plasma IgA/IgG ratios are higher in infected than in uninfected vaccine recipients in RV144. This suggests including relative IgA/IgG responses as an endpoint for characterizing pox-protein HIV vaccine candidates.
12. Zolla-Pazner S, deCamp A, Gilbert PB, Williams C, Yates NL, Williams WT, Howington R, Fong Y, Morris DE, Soderberg KA, et al. Vaccine-induced IgG antibodies to V1V2 regions of multiple HIV-1 subtypes correlate with decreased risk of HIV-1 infection. PloS One. 2014;9(2):e87572. [PubMed] (*) This paper reported that the IgG V2 antibody correlate of low HIV-1 infection risk observed in RV144 (Haynes et al., 2012) was reproduced in a follow-up study with multiple V2 antigens and two assays in two labs. This study also demonstrated that the V2 IgG correlate targeted a breadth of cross-clade V2 sequences. This replication provides some statistical robustness to the original V2 correlate of risk finding of Haynes et al. (2012).
13. Tomaras GD, Haynes BF. Advancing toward HIV-1 vaccine efficacy through the intersections of immune correlates. Vaccines. 2014;2(1):15–35. [PMC free article] [PubMed]
14. Lin L, Finak G, Ushey K, Seshadri C, Hawn TR, Frahm N, Scriba TJ, Mahomed H, Hanekom W, Bart PA, et al. COMPASS identifies T-cell subsets correlated with clinical outcomes. Nature Biotechnology. 2015;33(6):610–616. [PMC free article] [PubMed] (*) This paper developed a new statistical approach for measuring T cell polyfunctionality with a polyfunctionality score, and showed that this score for CD4 T cells in RV144 was a better predictor of low HIV-1 infection risk than magnitude of CD4 T cell response. This paper supports further study of the quality of CD4 T cells as a correlate of protection for vaccines against viral diseases.
15. Corey L, Gilbert PB, Tomaras G, Haynes BF, Pantaleo G, Fauci AS. Immune correlates of vaccine protection against HIV-1 acquisition: A review. Sci Transl Med. 2015 In Press. [PMC free article] [PubMed] (**) This review paper summarizes evidence from follow-up research about whether the original IgG V2 correlate of risk finding from Haynes et al. (2012) could also be a correlate of protection. The research has been supportive, with outcomes: (1) replication of the correlate result with cross-clade V2 antigens with two assays and two labs; (2) a viral sieve analysis showed differential vaccine efficacy at amino acid sites of IgG V2 loop reactivity; and (3) crystal structures and alanine scanning of V2 loop monoclonal antibodies from RV144 infected participants showed direct immune pressure on V2 mediated by ADCC. This synthesis motivates future efficacy testing of vaccines with putatively improved IgG V2 antibody responses.
16. Yates NL, Liao HX, Fong Y, deCamp A, Vandergrift NA, Williams WT, Alam SM, Ferrari G, Yang Z, Seaton KE, et al. Vaccine-induced Env V1-V2 IgG3 correlates with lower HIV-1 infection risk and declines soon after vaccination. Science Translational Medicine. 2014;6(228):228ra39-228ra39. [PMC free article] [PubMed] (*) This paper noted that Env V1V2-specific IgG3 was the immunoglobulin subclass showing the strongest inverse correlation low HIV-1 infection risk in RV144.
17. Chung AW, Ghebremichael M, Robinson H, Brown E, Choi I, Lane S, Dugast A, Shoen MK, Rolland M, Suscovich TJ, et al. Polyfunctional Fc-effector profiles mediated by IgG subclass selection distinguish RV144 and VAX003 vaccines. Science Translational Medicine. 2014;6(228):228ra38-228ra38. [PubMed] (*) This paper demonstrated that the IgG3 subclass engaged Fc-mediated effector responses more effectively than other IgG subclasses, thereby providing a possible mechanism explaining the association of Env V1V2 IgG3 with a lower rate of HIV acquisition.
18. Buchbinder SP, Mehrotra DV, Duerr A, Fitzgerald DW, Mogg R, Li D, Gilbert PB, Lama JR, Marmor M, Del Rio C, et al. Efficacy assessment of a cell-mediated immunity HIV-1 vaccine (the Step Study): a double-blind, randomised, placebo-controlled, test-of-concept trial. The Lancet. 2008;372(9653):1881–1893. [PMC free article] [PubMed]
19. Gray GE, Allen M, Moodie Z, Churchyard G, Bekker LG, Nchabeleng M, Mlisana K, Metch B, de Bruyn G, Latka MH, et al. Safety and efficacy of the HVTN 503/Phambili study of a clade-B-based HIV-1 vaccine in South Africa: a double-blind, randomised, placebo-controlled test-of-concept phase 2b study. The Lancet Infectious Diseases. 2011;11(7):507–515. [PMC free article] [PubMed]
20. Hammer SM, Sobieszczyk ME, Janes H, Karuna ST, Mulligan MJ, Grove D, Koblin BA, Buchbinder SP, Keefer MC, Tomaras GD, et al. Efficacy trial of a DNA/rAd5 HIV-1 preventive vaccine. N Engl J Med. 2013;369(22):2083–2092. [PMC free article] [PubMed]
21. Corey L, Nabel GJ, Dieffenbach C, Gilbert PB, Haynes BF, Johnston M, Kublin J, Lane HC, Pantaleo G, Picker LJ, et al. HIV-1 vaccines and adaptive trial designs. Sci Transl Med. 2011;3(79):79ps13. [PMC free article] [PubMed]
22. Gilbert PB, Grove D, Gabriel E, Huang Y, Gray G, Hammer SM, Buchbinder SP, Kublin J, Corey L, Self SG. A sequential phase 2b trial design for evaluating vaccine efficacy and immune correlates for multiple HIV vaccine regimens. Stat Commun Infect Dis. 2011;3(1) [PMC free article] [PubMed]
23. Lewis GK, DeVico AL, Gallo RC. Antibody persistence and T-cell balance: two key factors confronting HIV vaccine development. Proc Natl Acad Sci USA. 2014;111(44):15614–15621. [PubMed] (*) Authors in this paper discussed the evidence illustrating the poor persistence of antibody responses to Env and the related problem of CD4+ T-cell responses that compromise vaccine efficacy by creating excess celllular targets of HIV-1 infection, and proposed solutions to both problems applicable to Env-based AIDS vaccines.
24. Follmann D. Augmented designs to assess immune response in vaccine trials. Biometrics. 2006;62(4):1161–1169. [PMC free article] [PubMed]
25. Gilbert PB, Hudgens MG. Evaluating candidate principal surrogate endpoints. Biometrics. 2008;64(4):1146–1154. [PMC free article] [PubMed]
26. Huang Y, Gilbert PB, Wolfson J. Design and estimation for evaluating principal surrogate markers in vaccine trials. Biometrics. 2013;69(2):301–309. [PMC free article] [PubMed]
27. Manrique A, Adams E, Barouch DH, Fast P, Graham BS, Kim JH, Kublin JG, McCluskey M, Pantaleo G, Robinson HL, Russell N, Snow W, Johnston MI. The immune space: a concept and template for rationalizing vaccine development. AIDS Res Hum Retroviruses. 2014;30(11):1017–1022. [PubMed] (*) This paper proposed an ``immune space template" that provides a standardized approach by which the quality, level, and durability of immune responses elicited in early human trials by a candidate vaccine can be described.
28. Black P. "Manhattan distance". In: Pieterse Vreda, Black Paul E., editors. Dictionary of algorithms and data structures. 2006. [online]
29. Tibshirani R, Walther G, Hastie T. Estimating the number of clusters in a data set via the gap statistic. J. R. Stat. Soc., B. 2001;63(2):411–423.