Search tips
Search criteria

Results 1-25 (488376)

Clipboard (0)

Related Articles

1.  Principal Stratification in Causal Inference 
Biometrics  2002;58(1):21-29.
Many scientific problems require that treatment comparisons be adjusted for posttreatment variables, but the estimands underlying standard methods are not causal effects. To address this deficiency, we propose a general framework for comparing treatments adjusting for posttreatment variables that yields principal effects based on principal stratification. Principal stratification with respect to a posttreatment variable is a cross-classification of subjects defined by the joint potential values of that posttreatment variable under each of the treatments being compared. Principal effects are causal effects within a principal stratum. The key property of principal strata is that they are not affected by treatment assignment and therefore can be used just as any pretreatment covariate, such as age category. As a result, the central property of our principal effects is that they are always causal effects and do not suffer from the complications of standard posttreatment-adjusted estimands. We discuss briefly that such principal causal effects are the link between three recent applications with adjustment for posttreatment variables: (i) treatment noncompliance, (ii) missing outcomes (dropout) following treatment noncompliance, and (iii) censoring by death. We then attack the problem of surrogate or biomarker endpoints, where we show, using principal causal effects, that all current definitions of surrogacy, even when perfectly true, do not generally have the desired interpretation as causal effects of treatment on outcome. We go on to formulate estimands based on principal stratification and principal causal effects and show their superiority.
PMCID: PMC4137767  PMID: 11890317
Biomarker; Causal inference; Censoring by death; Missing data; Posttreatment variable; Principal stratification; Quality of life; Rubin causal model; Surrogate
The annals of applied statistics  2008;2(1):386-407.
Assessing immune responses to study vaccines as surrogates of protection plays a central role in vaccine clinical trials. Motivated by three ongoing or pending HIV vaccine efficacy trials, we consider such surrogate endpoint assessment in a randomized placebo-controlled trial with case-cohort sampling of immune responses and a time to event endpoint. Based on the principal surrogate definition under the principal stratification framework proposed by Frangakis and Rubin [Biometrics 58 (2002) 21–29] and adapted by Gilbert and Hudgens (2006), we introduce estimands that measure the value of an immune response as a surrogate of protection in the context of the Cox proportional hazards model. The estimands are not identified because the immune response to vaccine is not measured in placebo recipients. We formulate the problem as a Cox model with missing covariates, and employ novel trial designs for predicting the missing immune responses and thereby identifying the estimands. The first design utilizes information from baseline predictors of the immune response, and bridges their relationship in the vaccine recipients to the placebo recipients. The second design provides a validation set for the unmeasured immune responses of uninfected placebo recipients by immunizing them with the study vaccine after trial closeout. A maximum estimated likelihood approach is proposed for estimation of the parameters. Simulated data examples are given to evaluate the proposed designs and study their properties.
PMCID: PMC2601643  PMID: 19079758
Clinical trial; discrete failure time model; missing data; potential outcomes; principal stratification; surrogate marker
3.  Statistical identifiability and the surrogate endpoint problem, with application to vaccine trials 
Biometrics  2010;66(4):1153-1161.
Given a randomized treatment Z, a clinical outcome Y, and a biomarker S measured some fixed time after Z is administered, we may be interested in addressing the surrogate endpoint problem by evaluating whether S can be used to reliably predict the effect of Z on Y. Several recent proposals for the statistical evaluation of surrogate value have been based on the framework of principal stratification. In this paper, we consider two principal stratification estimands: joint risks and marginal risks. Joint risks measure causal associations of treatment effects on S and Y, providing insight into the surrogate value of the biomarker, but are not statistically identifiable from vaccine trial data. While marginal risks do not measure causal associations of treatment effects, they nevertheless provide guidance for future research, and we describe a data collection scheme and assumptions under which the marginal risks are statistically identifiable. We show how different sets of assumptions affect the identifiability of these estimands; in particular, we depart from previous work by considering the consequences of relaxing the assumption of no individual treatment effects on Y before S is measured. Based on algebraic relationships between joint and marginal risks, we propose a sensitivity analysis approach for assessment of surrogate value, and show that in many cases the surrogate value of a biomarker may be hard to establish, even when the sample size is large.
PMCID: PMC3597127  PMID: 20105158
Estimated likelihood; Identifiability; Principal stratification; Sensitivity analysis; Surrogate endpoint; Vaccine trials
4.  Causal Vaccine Effects on Binary Postinfection Outcomes 
The effects of vaccine on postinfection outcomes, such as disease, death, and secondary transmission to others, are important scientific and public health aspects of prophylactic vaccination. As a result, evaluation of many vaccine effects condition on being infected. Conditioning on an event that occurs posttreatment (in our case, infection subsequent to assignment to vaccine or control) can result in selection bias. Moreover, because the set of individuals who would become infected if vaccinated is likely not identical to the set of those who would become infected if given control, comparisons that condition on infection do not have a causal interpretation. In this article we consider identifiability and estimation of causal vaccine effects on binary postinfection outcomes. Using the principal stratification framework, we define a postinfection causal vaccine efficacy estimand in individuals who would be infected regardless of treatment assignment. The estimand is shown to be not identifiable under the standard assumptions of the stable unit treatment value, monotonicity, and independence of treatment assignment. Thus selection models are proposed that identify the causal estimand. Closed-form maximum likelihood estimators (MLEs) are then derived under these models, including those assuming maximum possible levels of positive and negative selection bias. These results show the relations between the MLE of the causal estimand and two commonly used estimators for vaccine effects on postinfection outcomes. For example, the usual intent-to-treat estimator is shown to be an upper bound on the postinfection causal vaccine effect provided that the magnitude of protection against infection is not too large. The methods are used to evaluate postinfection vaccine effects in a clinical trial of a rotavirus vaccine candidate and in a field study of a pertussis vaccine. Our results show that pertussis vaccination has a significant causal effect in reducing disease severity.
PMCID: PMC2603579  PMID: 19096723
Causal inference; Infectious disease; Maximum likelihood; Principal stratification; Sensitivity analysis
5.  Evaluating Candidate Principal Surrogate Endpoints 
Biometrics  2008;64(4):1146-1154.
Frangakis and Rubin (2002, Biometrics 58, 21–29) proposed a new definition of a surrogate endpoint (a “principal” surrogate) based on causal effects. We introduce an estimand for evaluating a principal surrogate, the causal effect predictiveness (CEP) surface, which quantifies how well causal treatment effects on the biomarker predict causal treatment effects on the clinical endpoint. Although the CEP surface is not identifiable due to missing potential outcomes, it can be identified by incorporating a baseline covariate(s) that predicts the biomarker. Given case–cohort sampling of such a baseline predictor and the biomarker in a large blinded randomized clinical trial, we develop an estimated likelihood method for estimating the CEP surface. This estimation assesses the “surrogate value” of the biomarker for reliably predicting clinical treatment effects for the same or similar setting as the trial. A CEP surface plot provides a way to compare the surrogate value of multiple biomarkers. The approach is illustrated by the problem of assessing an immune response to a vaccine as a surrogate endpoint for infection.
PMCID: PMC2726718  PMID: 18363776
Case cohort; Causal inference; Clinical trial; HIV vaccine; Postrandomization selection bias; Structural model; Prentice criteria; Principal stratification
6.  Hard, harder, hardest: principal stratification, statistical identifiability, and the inherent difficulty of finding surrogate endpoints 
In many areas of clinical investigation there is great interest in identifying and validating surrogate endpoints, biomarkers that can be measured a relatively short time after a treatment has been administered and that can reliably predict the effect of treatment on the clinical outcome of interest. However, despite dramatic advances in the ability to measure biomarkers, the recent history of clinical research is littered with failed surrogates. In this paper, we present a statistical perspective on why identifying surrogate endpoints is so difficult. We view the problem from the framework of causal inference, with a particular focus on the technique of principal stratification (PS), an approach which is appealing because the resulting estimands are not biased by unmeasured confounding. In many settings, PS estimands are not statistically identifiable and their degree of non-identifiability can be thought of as representing the statistical difficulty of assessing the surrogate value of a biomarker. In this work, we examine the identifiability issue and present key simplifying assumptions and enhanced study designs that enable the partial or full identification of PS estimands. We also present example situations where these assumptions and designs may or may not be feasible, providing insight into the problem characteristics which make the statistical evaluation of surrogate endpoints so challenging.
PMCID: PMC4171402  PMID: 25342953
Surrogate endpoint; Principal stratification; Causal inference; Statistical identifiability
7.  Causal Inference for Vaccine Effects on Infectiousness 
The International Journal of Biostatistics  2012;8(2):10.2202/1557-4679.1354 /j/ijb.2012.8.issue-2/1557-4679.1354/1557-4679.1354.xml.
If a vaccine does not protect individuals completely against infection, it could still reduce infectiousness of infected vaccinated individuals to others. Typically, vaccine efficacy for infectiousness is estimated based on contrasts between the transmission risk to susceptible individuals from infected vaccinated individuals compared with that from infected unvaccinated individuals. Such estimates are problematic, however, because they are subject to selection bias and do not have a causal interpretation. Here, we develop causal estimands for vaccine efficacy for infectiousness for four different scenarios of populations of transmission units of size two. These causal estimands incorporate both principal stratification, based on the joint potential infection outcomes under vaccine and control, and interference between individuals within transmission units. In the most general scenario, both individuals can be exposed to infection outside the transmission unit and both can be assigned either vaccine or control. The three other scenarios are special cases of the general scenario where only one individual is exposed outside the transmission unit or can be assigned vaccine. The causal estimands for vaccine efficacy for infectiousness are well defined only within certain principal strata and, in general, are identifiable only with strong unverifiable assumptions. Nonetheless, the observed data do provide some information, and we derive large sample bounds on the causal vaccine efficacy for infectiousness estimands. An example of the type of data observed in a study to estimate vaccine efficacy for infectiousness is analyzed in the causal inference framework we developed.
PMCID: PMC3348179  PMID: 22499732
causal inference; principal stratification; interference; infectious disease; vaccine
8.  Sensitivity Analysis of Per-Protocol Time-to-Event Treatment Efficacy in Randomized Clinical Trials 
Journal of the American Statistical Association  2013;108(503):10.1080/01621459.2013.786649.
Assessing per-protocol treatment effcacy on a time-to-event endpoint is a common objective of randomized clinical trials. The typical analysis uses the same method employed for the intention-to-treat analysis (e.g., standard survival analysis) applied to the subgroup meeting protocol adherence criteria. However, due to potential post-randomization selection bias, this analysis may mislead about treatment efficacy. Moreover, while there is extensive literature on methods for assessing causal treatment effects in compliers, these methods do not apply to a common class of trials where a) the primary objective compares survival curves, b) it is inconceivable to assign participants to be adherent and event-free before adherence is measured, and c) the exclusion restriction assumption fails to hold. HIV vaccine efficacy trials including the recent RV144 trial exemplify this class, because many primary endpoints (e.g., HIV infections) occur before adherence is measured, and nonadherent subjects who receive some of the planned immunizations may be partially protected. Therefore, we develop methods for assessing per-protocol treatment efficacy for this problem class, considering three causal estimands of interest. Because these estimands are not identifiable from the observable data, we develop nonparametric bounds and semiparametric sensitivity analysis methods that yield estimated ignorance and uncertainty intervals. The methods are applied to RV144.
PMCID: PMC3811958  PMID: 24187408
As-treated; Bounds; Causal inference; Exclusion restriction; Ignorance region; Intention to treat; Principal stratification; Selection bias; Survival analysis
9.  Surrogacy assessment using principal stratification when surrogate and outcome measures are multivariate normal 
Biostatistics (Oxford, England)  2013;15(2):266-283.
In clinical trials, a surrogate outcome variable (S) can be measured before the outcome of interest (T) and may provide early information regarding the treatment (Z) effect on T. Using the principal surrogacy framework introduced by Frangakis and Rubin (2002. Principal stratification in causal inference. Biometrics 58, 21–29), we consider an approach that has a causal interpretation and develop a Bayesian estimation strategy for surrogate validation when the joint distribution of potential surrogate and outcome measures is multivariate normal. From the joint conditional distribution of the potential outcomes of T, given the potential outcomes of S, we propose surrogacy validation measures from this model. As the model is not fully identifiable from the data, we propose some reasonable prior distributions and assumptions that can be placed on weakly identified parameters to aid in estimation. We explore the relationship between our surrogacy measures and the surrogacy measures proposed by Prentice (1989. Surrogate endpoints in clinical trials: definition and operational criteria. Statistics in Medicine 8, 431–440). The method is applied to data from a macular degeneration study and an ovarian cancer study.
PMCID: PMC4023321  PMID: 24285772
Bayesian estimation; Principal stratification; Surrogate endpoints
10.  Surrogacy Assessment Using Principal Stratification With Multivariate Normal and Gaussian Copula Models 
Clinical trials (London, England)  2014;12(4):317-322.
The validation of intermediate markers as surrogate markers (S) for the true outcome of interest (T) in clinical trials offers the possibility for trials to be run more quickly and cheaply by using the surrogate endpoint in place of the true endpoint.
Working within a principal stratification framework, we propose causal quantities to evaluate surrogacy using a Gaussian copula model for an ordinal surrogate and time-to-event final outcome. The methods are applied to data from four colorectal cancer clinical trials where S is tumor response and T is overall survival.
For the Gaussian copula model, a Bayesian estimation strategy is used and, as some parameters are not identifiable from the data, we explore the use of informative priors that are consistent with reasonable assumptions in the surrogate marker setting to aid in estimation.
While there is some bias in the estimation of the surrogacy quantities of interest, the estimation procedure does reasonably well at distinguishing between poor and good surrogate markers.
Some of the parameters of the proposed model are not identifiable from the data, and therefore assumptions must be made in order to aid in their estimation.
The proposed quantities can be used in combination to provide evidence about the validity of S as a surrogate marker for T.
PMCID: PMC4768476  PMID: 25490988
Bayesian estimation; Causal inference; Gaussian copula; Potential outcomes; Surrogate endpoint
11.  Partially hidden Markov model for time-varying principal stratification in HIV prevention trials 
It is frequently of interest to estimate the intervention effect that adjusts for post-randomization variables in clinical trials. In the recently completed HPTN 035 trial, there is differential condom use between the three microbicide gel arms and the No Gel control arm, so that intention to treat (ITT) analyses only assess the net treatment effect that includes the indirect treatment effect mediated through differential condom use. Various statistical methods in causal inference have been developed to adjust for post-randomization variables. We extend the principal stratification framework to time-varying behavioral variables in HIV prevention trials with a time-to-event endpoint, using a partially hidden Markov model (pHMM). We formulate the causal estimand of interest, establish assumptions that enable identifiability of the causal parameters, and develop maximum likelihood methods for estimation. Application of our model on the HPTN 035 trial reveals an interesting pattern of prevention effectiveness among different condom-use principal strata.
PMCID: PMC3649016  PMID: 23667279
microbicide; causal inference; posttreatment variables; direct effect
12.  Principal Stratification and Attribution Prohibition: Good Ideas Taken Too Far 
Pearl’s article provides a useful springboard for discussing further the benefits and drawbacks of principal stratification and the associated discomfort with attributing effects to post-treatment variables. The basic insights of the approach are important: pay close attention to modification of treatment effects by variables not observable before treatment decisions are made, and be careful in attributing effects to variables when counterfactuals are ill-defined. These insights have often been taken too far in many areas of application of the approach, including instrumental variables, censoring by death, and surrogate outcomes. A novel finding is that the usual principal stratification estimand in the setting of censoring by death is by itself of little practical value in estimating intervention effects.
PMCID: PMC3204670  PMID: 22049269
principal stratification; causal inference
13.  Accommodating Missingness When Assessing Surrogacy Via Principal Stratification 
Clinical trials (London, England)  2013;10(3):363-377.
When an outcome of interest in a clinical trial is late-occurring or difficult to obtain, surrogate markers can extract information about the effect of the treatment on the outcome of interest. Understanding associations between the causal effect of treatment on the outcome and the causal effect of treatment on the surrogate is critical to understanding the value of a surrogate from a clinical perspective.
Traditional regression approaches to determine the proportion of the treatment effect explained by surrogate markers suffer from several shortcomings: they can be unstable, and can lie outside of the 0–1 range. Further, they do not account for the fact that surrogate measures are obtained post-randomization, and thus the surrogate-outcome relationship may be subject to unmeasured confounding. Methods to avoid these problem are of key importance.
Frangakis C, Rubin DM. Principal stratification in causal inference. Biometrics 2002; 58:21–9 suggested assessing the causal effect of treatment within pre-randomization “principal strata” defined by the counterfactual joint distribution of the surrogate marker under the different treatment arms, with the proportion of the overall outcome causal effect attributable to subjects for whom the treatment affects the proposed surrogate as the key measure of interest. Li Y, Taylor JMG, Elliott MR. Bayesian approach to surrogacy assessment using principal stratification in clinical trials. Biometrics 2010; 66:523–31 developed this “principal surrogacy” approach for dichotomous markers and outcomes, utilizing Bayesian methods that accommodated non-identifiability in the model parameters. Because the surrogate marker is typically observed early, outcome data is often missing. Here we extend Li, Taylor, and Elliott to accommodate missing data in the observable final outcome under ignorable and non-ignorable settings. We also allow for the possibility that missingness has a counterfactual component, a feature that previous literature has not addressed.
We apply the proposed methods to a trial of glaucoma control comparing surgery versus medication, where intraocular pressure (IOP) control at 12 months is a surrogate for IOP control at 96 months. We also conduct a series of simulations to consider the impacts of non-ignorability, as well as sensitivity to priors and the ability of the Decision Information Criterion to choose the correct model when parameters are not fully identified.
Because model parameters cannot be fully identified from data, informative priors can introduce non-trivial bias in moderate sample size settings, while more non-informative priors can yield wide credible intervals.
Assessing the linkage between causal effects of treatment on a surrogate marker and causal effects of a treatment on an outcome is important to understanding the value of a marker. These causal effects are not fully identifiable: hence we explore the sensitivity and identifiability aspects of these models and show that relatively weak assumptions can still yield meaningful results.
PMCID: PMC4096330  PMID: 23553326
Causal Inference; Surrogate Marker; Bayesian Analysis; dentifiability; Non-response; Counterfactual
14.  Estimating Causal Effects in Trials Involving Multi-Treatment Arms Subject to Non-compliance: A Bayesian framework 
Data analysis for randomized trials including multi-treatment arms is often complicated by subjects who do not comply with their treatment assignment. We discuss here methods of estimating treatment efficacy for randomized trials involving multi-treatment arms subject to non-compliance. One treatment effect of interest in the presence of non-compliance is the complier average causal effect (CACE) (Angrist et al. 1996), which is defined as the treatment effect for subjects who would comply regardless of the assigned treatment. Following the idea of principal stratification (Frangakis & Rubin 2002), we define principal compliance (Little et al. 2009) in trials with three treatment arms, extend CACE and define causal estimands of interest in this setting. In addition, we discuss structural assumptions needed for estimation of causal effects and the identifiability problem inherent in this setting from both a Bayesian and a classical statistical perspective. We propose a likelihood-based framework that models potential outcomes in this setting and a Bayes procedure for statistical inference. We compare our method with a method of moments approach proposed by Cheng & Small (2006) using a hypothetical data set, and further illustrate our approach with an application to a behavioral intervention study (Janevic et al. 2003).
PMCID: PMC3104736  PMID: 21637737
Causal Inference; Complier Average Causal Effect; Multi-arm Trials; Non-compliance; Principal Compliance; Principal Stratification
15.  Surrogacy Assessment Using Principal Stratification and a Gaussian Copula Model 
In clinical trials, a surrogate outcome (S) can be measured before the outcome of interest (T) and may provide early information regarding the treatment (Z) effect on T. Many methods of surrogacy validation rely on models for the conditional distribution of T given Z and S. However, S is a post-randomization variable, and unobserved, simultaneous predictors of S and T may exist, resulting in a non-causal interpretation. Frangakis and Rubin1 developed the concept of principal surrogacy, stratifying on the joint distribution of the surrogate marker under treatment and control to assess the association between the causal effects of treatment on the marker and the causal effects of treatment on the clinical outcome. Working within the principal surrogacy framework, we address the scenario of an ordinal categorical variable as a surrogate for a censored failure time true endpoint. A Gaussian copula model is used to model the joint distribution of the potential outcomes of T, given the potential outcomes of S. Because the proposed model cannot be fully identified from the data, we use a Bayesian estimation approach with prior distributions consistent with reasonable assumptions in the surrogacy assessment setting. The method is applied to data from a colorectal cancer clinical trial, previously analyzed by Burzykowski et al..2
PMCID: PMC4272338  PMID: 24947559
Causal inference; Gaussian copula; Potential outcomes; Surrogate endpoint
16.  Design and Estimation for Evaluating Principal Surrogate Markers in Vaccine Trials 
Biometrics  2013;69(2):301-309.
In vaccine research, immune biomarkers that can reliably predict a vaccine’s effect on the clinical endpoint (i.e., surrogate markers) are important tools for guiding vaccine development. This paper addresses issues on optimizing two-phase sampling study design for evaluating surrogate markers in a principal surrogate framework, motivated by the design of a future HIV vaccine trial. To address the problem of missing potential outcomes in a standard trial design, novel trial designs have been proposed that utilize baseline predictors of the immune response biomarker(s) and/or augment the trial by vaccinating uninfected placebo recipients at the end of the trial and measuring their immune biomarkers. However, inefficient use of the augmented information can lead to counterintuitive results on the precision of estimation. To remedy this problem, we propose a pseudo-score type estimator suitable for the augmented design and characterize its asymptotic properties. This estimator has superior performance compared with existing estimators and allows calculation of analytical variances useful for guiding study design. Based on the new estimator we investigate in detail the problem of optimizing the sampling scheme of a biomarker in a vaccine efficacy trial for efficiently estimating its surrogate effect, as characterized by the vaccine efficacy curve (a causal effect predictiveness curve) and by the predicted overall vaccine efficacy using the biomarker.
PMCID: PMC3713795  PMID: 23409839
Closeout placebo vaccination; Estimated likelihood; Immune correlate; Principal surrogate; Pseudo-score; Two-phase sampling design
17.  Is blood pressure reduction a valid surrogate endpoint for stroke prevention? an analysis incorporating a systematic review of randomised controlled trials, a by-trial weighted errors-in-variables regression, the surrogate threshold effect (STE) and the biomarker-surrogacy (BioSurrogate) evaluation schema (BSES) 
Blood pressure is considered to be a leading example of a valid surrogate endpoint. The aims of this study were to (i) formally evaluate systolic and diastolic blood pressure reduction as a surrogate endpoint for stroke prevention and (ii) determine what blood pressure reduction would predict a stroke benefit.
We identified randomised trials of at least six months duration comparing any pharmacologic anti-hypertensive treatment to placebo or no treatment, and reporting baseline blood pressure, on-trial blood pressure, and fatal and non-fatal stroke. Trials with fewer than five strokes in at least one arm were excluded. Errors-in-variables weighted least squares regression modelled the reduction in stroke as a function of systolic blood pressure reduction and diastolic blood pressure reduction respectively. The lower 95% prediction band was used to determine the minimum systolic blood pressure and diastolic blood pressure difference, the surrogate threshold effect (STE), below which there would be no predicted stroke benefit. The STE was used to generate the surrogate threshold effect proportion (STEP), a surrogacy metric, which with the R-squared trial-level association was used to evaluate blood pressure as a surrogate endpoint for stroke using the Biomarker-Surrogacy Evaluation Schema (BSES3).
In 18 qualifying trials representing all pharmacologic drug classes of antihypertensives, assuming a reliability coefficient of 0.9, the surrogate threshold effect for a stroke benefit was 7.1 mmHg for systolic blood pressure and 2.4 mmHg for diastolic blood pressure. The trial-level association was 0.41 and 0.64 and the STEP was 66% and 78% for systolic and diastolic blood pressure respectively. The STE and STEP were more robust to measurement error in the independent variable than R-squared trial-level associations. Using the BSES3, assuming a reliability coefficient of 0.9, systolic blood pressure was a B + grade and diastolic blood pressure was an A grade surrogate endpoint for stroke prevention. In comparison, using the same stroke data sets, no STEs could be estimated for cardiovascular (CV) mortality or all-cause mortality reduction, although the STE for CV mortality approached 25 mmHg for systolic blood pressure.
In this report we provide the first surrogate threshold effect (STE) values for systolic and diastolic blood pressure. We suggest the STEs have face and content validity, evidenced by the inclusivity of trial populations, subject populations and pharmacologic intervention populations in their calculation. We propose that the STE and STEP metrics offer another method of evaluating the evidence supporting surrogate endpoints. We demonstrate how surrogacy evaluations are strengthened if formally evaluated within specific-context evaluation frameworks using the Biomarker- Surrogate Evaluation Schema (BSES3), and we discuss the implications of our evaluation of blood pressure on other biomarkers and patient-reported instruments in relation to surrogacy metrics and trial design.
PMCID: PMC3388460  PMID: 22409774
Blood pressure; Stroke; Surrogate Endpoint; Biomarker
18.  Clarifying the Role of Principal Stratification in the Paired Availability Design 
The paired availability design for historical controls postulated four classes corresponding to the treatment (old or new) a participant would receive if arrival occurred during either of two time periods associated with different availabilities of treatment. These classes were later extended to other settings and called principal strata. Judea Pearl asks if principal stratification is a goal or a tool and lists four interpretations of principal stratification. In the case of the paired availability design, principal stratification is a tool that falls squarely into Pearl's interpretation of principal stratification as “an approximation to research questions concerning population averages.” We describe the paired availability design and the important role played by principal stratification in estimating the effect of receipt of treatment in a population using data on changes in availability of treatment. We discuss the assumptions and their plausibility. We also introduce the extrapolated estimate to make the generalizability assumption more plausible. By showing why the assumptions are plausible we show why the paired availability design, which includes principal stratification as a key component, is useful for estimating the effect of receipt of treatment in a population. Thus, for our application, we answer Pearl's challenge to clearly demonstrate the value of principal stratification.
PMCID: PMC3114955  PMID: 21686085
principal stratification; causal inference; paired availability design
19.  Causal assessment of surrogacy in a meta-analysis of colorectal cancer trials 
Biostatistics (Oxford, England)  2011;12(3):478-492.
When the true end points (T) are difficult or costly to measure, surrogate markers (S) are often collected in clinical trials to help predict the effect of the treatment (Z). There is great interest in understanding the relationship among S, T, and Z. A principal stratification (PS) framework has been proposed by Frangakis and Rubin (2002) to study their causal associations. In this paper, we extend the framework to a multiple trial setting and propose a Bayesian hierarchical PS model to assess surrogacy. We apply the method to data from a large collection of colon cancer trials in which S and T are binary. We obtain the trial-specific causal measures among S, T, and Z, as well as their overall population-level counterparts that are invariant across trials. The method allows for information sharing across trials and reduces the nonidentifiability problem. We examine the frequentist properties of our model estimates and the impact of the monotonicity assumption using simulations. We also illustrate the challenges in evaluating surrogacy in the counterfactual framework that result from nonidentifiability.
PMCID: PMC3114655  PMID: 21252079
Bayesian estimation; Counterfactual model; Identifiability; Multiple trials; Principal stratification; Surrogate marker
20.  Principal Stratification — Uses and Limitations 
Pearl (2011) asked for the causal inference community to clarify the role of the principal stratification framework in the analysis of causal effects. Here, I argue that the notion of principal stratification has shed light on problems of non-compliance, censoring-by-death, and the analysis of post-infection outcomes; that it may be of use in considering problems of surrogacy but further development is needed; that it is of some use in assessing “direct effects”; but that it is not the appropriate tool for assessing “mediation.” There is nothing within the principal stratification framework that corresponds to a measure of an “indirect” or “mediated” effect.
PMCID: PMC3154088  PMID: 21841939
causal inference; mediation; non-compliance; potential outcomes; principal stratification; surrogates
21.  Broad Blockade Antibody Responses in Human Volunteers after Immunization with a Multivalent Norovirus VLP Candidate Vaccine: Immunological Analyses from a Phase I Clinical Trial 
PLoS Medicine  2015;12(3):e1001807.
Human noroviruses (NoVs) are the primary cause of acute gastroenteritis and are characterized by antigenic variation between genogroups and genotypes and antigenic drift of strains within the predominant GII.4 genotype. In the context of this diversity, an effective NoV vaccine must elicit broadly protective immunity. We used an antibody (Ab) binding blockade assay to measure the potential cross-strain protection provided by a multivalent NoV virus-like particle (VLP) candidate vaccine in human volunteers.
Methods and Findings
Sera from ten human volunteers immunized with a multivalent NoV VLP vaccine (genotypes GI.1/GII.4) were analyzed for IgG and Ab blockade of VLP interaction with carbohydrate ligand, a potential correlate of protective immunity to NoV infection and illness. Immunization resulted in rapid rises in IgG and blockade Ab titers against both vaccine components and additional VLPs representing diverse strains and genotypes not represented in the vaccine. Importantly, vaccination induced blockade Ab to two novel GII.4 strains not in circulation at the time of vaccination or sample collection. GII.4 cross-reactive blockade Ab titers were more potent than responses against non-GII.4 VLPs, suggesting that previous exposure history to this dominant circulating genotype may impact the vaccine Ab response. Further, antigenic cartography indicated that vaccination preferentially activated preexisting Ab responses to epitopes associated with GII.4.1997. Study interpretations may be limited by the relevance of the surrogate neutralization assay and the number of immunized participants evaluated.
Vaccination with a multivalent NoV VLP vaccine induces a broadly blocking Ab response to multiple epitopes within vaccine and non-vaccine NoV strains and to novel antigenic variants not yet circulating at the time of vaccination. These data reveal new information about complex NoV immune responses to both natural exposure and to vaccination, and support the potential feasibility of an efficacious multivalent NoV VLP vaccine for future use in human populations.
Trial Registration NCT01168401
Lisa Lindesmith and colleagues assess the potential of a candidate virus-like particle (VLP) vaccine to induce antibody responses to antigenically divergent norovirus strains.
Editors' Summary
Worldwide, noroviruses cause one in five cases of viral gastroenteritis (often called stomach flu or winter vomiting disease), the symptoms of which include nausea, vomiting, and diarrhea. There is no specific treatment for infection with these highly contagious viruses, and no established approach to vaccine development. While most people recover from the symptoms of norovirus infection within a few days, young children and the elderly may become severely ill or die. An estimated annual 300 million cases of norovirus infection contribute to roughly 260,000 deaths, mostly among this vulnerable demographic and mostly in low-income countries. Like influenza viruses, many noroviruses are evolving via a process known as antigenic drift. Antigens are components of infectious agents (including viruses) that are recognized by antibodies, proteins that bind to and neutralize foreign invaders. Over time, noroviruses develop small changes in their antigens that allow them to escape from antibodies produced in response to earlier infections. Every two to four years, because of accumulated antigenic drift, a new strain of norovirus emerges to which the human population has no direct antibody immunity, and an outbreak occurs. Because vaccines usually contain a component of the infectious agent that stimulates immunity, antigenic drift complicates the process of vaccine development. To be worth the cost and effort, a norovirus vaccine must confer immunity against a diverse range of norovirus strains, ideally including strains beyond those represented within the vaccine itself.
Partly because there is not a reliable method for growing noroviruses in the laboratory, recent efforts have focused on developing candidate vaccines using virus-like particles (VLPs). VLPs are constructed from laboratory-generated molecules of the virus’s capsid (outer shell). These capsid proteins self-assemble into icosahedral VLPs, which resemble the viral shell. VLPs cannot infect people or cause illness, but because they contain viral antigens, they can induce the immune system to produce antibodies that may neutralize actual viruses. VLPs can also be used to study the antibodies that people produce in response to vaccination or infection.
Why Was This Study Done?
VLP-based vaccines are relatively new, and their capacity to elicit a broad immune response conferring protection to an evolving range of norovirus strains is not established. One VLP vaccine based on a single strain that circulates primarily in children conferred immunity to that strain. Another, multivalent (containing a mix of VLPs from more than one strain) VLP vaccine elicited antibody generation, but in a phase I clinical trial did not confer immunity to infection by a strain that had previously circulated globally. In the current study, the researchers explored two key questions using laboratory analysis of blood samples drawn from participants in that trial. First, they tested whether the vaccine elicits antibody responses to a broad range of norovirus strains, as antibody responses can provide clues to the potential for this type of vaccine to confer broad immunity in the future. Second, they investigated how preexisting exposure to noroviruses affects the immune system’s response to a vaccine—strategic information that could aid in future vaccine development.
What Did the Researchers Do and Find?
The researchers tested serum (blood without cells or clotting proteins; serum contains the antibodies generated by the immune system) collected from ten participants receiving one injection of the VLP vaccine followed by a second injection 28 days later. They analyzed the serum specimens for antibodies to vaccine VLPs and also to VLPs representing viruses that were not contained in the vaccine. They used two methods, both utilizing VLPs generated from 11 norovirus strains: a traditional method that assesses binding of serum antibodies to each of these VLPs, and a more recent method that assays the ability of antibodies to block the interaction of each VLP with a molecule on intestinal cells that binds to the virus (the gut epithelial ligand), enabling norovirus to enter and infect cells. Prior studies suggest that this latter assay may be a better proxy for actual immunity.
The researchers’ major finding is that a multivalent VLP vaccine (two VLPs representing four strains of norovirus: one from a subgroup called genotype GI.1 and another consensus VLP of three strains from the subgroup GII.4) can rapidly elicit serum antibodies that bind a range of vaccine and non-vaccine VLPs, and that block binding of these VLPs to the gut epithelial ligand. Notably, vaccine recipients also generated antibodies reactive to two novel VLPs representing human noroviruses that they could not have previously encountered, indicating that prior exposure to each norovirus strain was not required for the full antibody response following vaccination. However, based on an analysis of which specific epitopes (small regions on an antigen) the population of antibodies binds, the authors report that antibody responses to the vaccine prominently target epitopes of a 1997 strain of human GII.4 norovirus, and propose that exposure history does influence the antibody response.
What Do these Findings Mean?
These findings raise the possibility that the VLP vaccine may induce immunity not only to norovirus strains that have caused past outbreaks, but also to variants that have yet to enter the population—a necessary attribute given the antigenic drift observed among noroviruses. The study also indicates that VLP-induced antibody responses to norovirus are consistent with the “antigenic seniority” model, in which strains to which an individual was previously exposed influence the binding properties of a vaccine-induced antibody population. This latter finding may influence the design of future norovirus vaccines.
These results must be interpreted cautiously, particularly as they pertain to the potential for a norovirus vaccine to protect against natural infection. The study is small, and antibody binding and blocking assays may not replicate how the immune system of a vaccine recipient will respond to true norovirus infection. Additionally, the study participants were all adults aged 18 to 49 years, while a vaccine is most needed for young children (who account for the majority of severe infections) and the elderly (who are most likely to die from infection). Unlike the study participants, young children lack preexisting antibodies to norovirus. Older people are more likely to have been previously exposed to norovirus, but may show attenuated immune responses to vaccination. Adapting to the different immune responses of these two groups remains a central challenge to norovirus vaccine development.
Additional Information
Please access these websites via the online version of this summary at
The World Health Organization provides a comprehensive description of the disease burden from diarrheal disease
The MedlinePlus encyclopedia has a page on viral gastroenteritis (in English and Spanish)
The US Centers for Disease Control and Prevention provides information on disease trends and outbreaks
The US Department of Health and Human Services offers guidance for prevention based on food safety
A 2014 interview with Academic Editor Benjamin Lopman explores the difficulty of developing a norovirus vaccine
The authors have previously published findings on the evolution of norovirus strains in PLOS Medicine and have discussed the challenges of norovirus therapeutic design in PLOS Pathogens
PMCID: PMC4371888  PMID: 25803642
22.  A Bayesian Approach to Improved Estimation of Causal Effect Predictiveness for a Principal Surrogate Endpoint 
Biometrics  2012;68(3):10.1111/j.1541-0420.2011.01736.x.
The literature on potential outcomes has shown that traditional methods for characterizing surrogate endpoints in clinical trials based only on observed quantities can fail to capture causal relationships between treatments, surrogates, and outcomes. Building on the potential-outcomes formulation of a principal surrogate, we introduce a Bayesian method to estimate the Causal Effect Predictiveness (CEP) surface and quantify a candidate surrogate’s utility for reliably predicting clinical outcomes. In considering the full joint distribution of all potentially-observable quantities, our Bayesian approach has the following features. First, our approach illuminates implicit assumptions embedded in previously-used estimation strategies that have been shown to result in poor performance. Second, our approach provides tools for making explicit and scientifically-interpretable assumptions regarding associations about which observed data are not informative. Through simulations based on an HIV vaccine trial, we found that the Bayesian approach can produce estimates of the CEP surface with improved performance compared to previous methods. Third, our approach can extend principal-surrogate estimation beyond the previously-considered setting of a vaccine trial where the candidate surrogate is constant in one arm of the study. We illustrate this extension through an application to an AIDS therapy trial where the candidate surrogate varies in both treatment arms.
PMCID: PMC3860118  PMID: 22348277
Biomarker; Causal effect predictiveness; principal stratification; surrogate endpoint
23.  Evaluating a surrogate endpoint at three levels, with application to vaccine development 
Statistics in medicine  2008;27(23):4758-4778.
Identification of an immune response to vaccination that reliably predicts protection from clinically significant infection, i.e. an immunological surrogate endpoint, is a primary goal of vaccine research. Using this problem of evaluating an immunological surrogate as an illustration, we describe a hierarchy of three criteria for a valid surrogate endpoint and statistical analysis frameworks for evaluating them. Based on a placebo-controlled vaccine efficacy trial, the first level entails assessing the correlation of an immune response with a study endpoint in the study groups, and the second level entails evaluating an immune response as a surrogate for the study endpoint that can be used for predicting vaccine efficacy for a setting similar to that of the vaccine trial. We show that baseline covariates, innovative study design, and a potential outcomes formulation can be helpful for this assessment. The third level entails validation of a surrogate endpoint via meta-analysis, where the goal is to evaluate how well the immune response can be used to predict vaccine efficacy for new settings (building bridges). A simulated vaccine trial and two example vaccine trials are presented, one supporting that certain anti-influenza antibody levels are an excellent surrogate for influenza illness and another supporting that certain anti-HIV antibody levels are not useful as a surrogate for HIV infection.
PMCID: PMC2646675  PMID: 17979212
clinical trial; counterfactual; immune correlate; meta-analysis; potential outcomes; principal surrogate; statistical surrogate
24.  Fold Rise in Antibody Titers by Measured by Glycoprotein-Based Enzyme-Linked Immunosorbent Assay Is an Excellent Correlate of Protection for a Herpes Zoster Vaccine, Demonstrated via the Vaccine Efficacy Curve 
The Journal of Infectious Diseases  2014;210(10):1573-1581.
Background. The phase III Zostavax Efficacy and Safety Trial of 1 dose of licensed zoster vaccine (ZV; Zostavax; Merck) in 50–59-year-olds showed approximately 70% vaccine efficacy (VE) to reduce the incidence of herpes zoster (HZ). An objective of the trial was to assess immune response biomarkers measuring antibodies to varicella zoster virus (VZV) by glycoprotein-based enzyme-linked immunosorbent assay as correlates of protection (CoPs) against HZ.
Methods. The principal stratification vaccine efficacy curve framework for statistically evaluating immune response biomarkers as CoPs was applied. The VE curve describes how VE against the clinical end point (HZ) varies across participant subgroups defined by biomarker readout measuring vaccine-induced immune response. The VE curve was estimated using several subgroup definitions.
Results. The fold rise in VZV antibody titers from the time before immunization to 6 weeks after immunization was an excellent CoP, with VE increasing sharply with fold rise: VE was estimated at 0% for the subgroup with no rise and at 90% for the subgroup with 5.26-fold rise. In contrast, VZV antibody titers measured 6 weeks after immunization did not predict VE, with similar estimated VEs across titer subgroups.
Conclusions. The analysis illustrates the value of the VE curve framework for assessing immune response biomarkers as CoPs in vaccine efficacy trials.
Clinical Trials Registration. NCT00534248.
PMCID: PMC4215071  PMID: 24823623
causal inference; correlate of immunity; immune correlate of protection; principal stratification; signature of protection; statistical analysis; surrogate endpoint; vaccine efficacy trial
25.  Invited Commentary: Decomposing with a Lot of Supposing 
American Journal of Epidemiology  2010;172(12):1349-1351.
In this issue of the Journal, VanderWeele and Vansteelandt (Am J Epidemiol. 2010;172(12):1339–1348) provide simple formulae for estimation of direct and indirect effects using standard logistic regression when the exposure and outcome are binary, the mediator is continuous, and the odds ratio is the chosen effect measure. They also provide concisely stated lists of assumptions necessary for estimation of these effects, including various conditional independencies and homogeneity of exposure and mediator effects over covariate strata. They further suggest that this will allow effect decomposition in case-control studies if the sampling fractions and population outcome prevalence are known with certainty. In this invited commentary, the author argues that, in a well-designed case-control study in which the sampling fraction is known, it should not be necessary to rely on the odds ratio. The odds ratio has well-known deficiencies as a causal parameter, and its use severely complicates evaluation of confounding and effect homogeneity. Although VanderWeele and Vansteelandt propose that a rare disease assumption is not necessary for estimation of controlled direct effects using their approach, collapsibility concerns suggest otherwise when the goal is causal inference rather than merely measuring association. Moreover, their clear statement of assumptions necessary for the estimation of natural/pure effects suggests that these quantities will rarely be viable estimands in observational epidemiology.
PMCID: PMC3139971  PMID: 21036956
causal inference; conditional independence; confounding; decomposition; estimation; interaction; logistic regression; odds ratio

Results 1-25 (488376)