The above discussion indicates use of three distinct populations to develop a disease risk score: 1. in an alternative data set or in a time period prior to the current study, perhaps before introduction of a new therapy; 2, in the study population, but based on estimation of disease risk in the unexposed group only, akin to the Peters-Belson method; 3, in the entire study population, based on a model including indicators of exposure status, and then set this exposure status to zero for an individual’s predicted risk, as suggested by Miettinen. Each approach seeks to estimate a disease risk score that will be the most representative of the study population, and each has both strengths and limitations.
Estimation of a disease risk score using all subjects in the study population, based on a model with an indicator for exposure status, benefits from the ready availability of the data set and its use of a larger sample size than estimation restricted to the unexposed, to yield potentially more reliable estimates of disease risk under the assumption of a correct model form. Several simulation studies have found that stratification on a disease risk score obtained in this way (according to the suggestion of Miettinen) performs comparably to both propensity score stratification and multivariable analysis, as long as covariates are not too highly correlated with exposure.30–33
Further within the context of the scenarios examined, this full-cohort disease risk score can sometimes outperform a disease risk score estimated in the unexposed subjects only.33
However, as pointed out by Hansen, the validity of the disease risk score estimated in this way is sensitive to model form, especially the assumption of a uniform treatment effect across categories of disease risk. Even modest treatment effect heterogeneity can induce bias in the overall treatment effect with this approach. If the treatment groups differ substantially on important covariates (which is akin to a clear distinction of treatment groups by means of a propensity score), these concerns are enhanced. Further, inclusion of the exposure effect in the estimation of the disease risk score limits its value as a balancing score, as also discussed by Hansen.
Estimation of the disease risk score among unexposed subjects in the study population is also readily implementable, makes fewer assumptions than standard approaches that include exposed subjects, and yields a balancing score with desirable theoretical properties. However, reliable estimation of the model is a particular challenge in settings with relatively few outcomes, and these are expected among the unexposed subjects in the early monitoring period for a new therapy. Further, if the disease risk score is used to form strata for estimation of treatment effects within levels of disease risk, the over-fitting of the model under this approach tends to over-estimate treatment benefits in the high-risk group and under-estimate treatment harms in the low-risk group, which substantially limits the value of the score as an axis upon which to evaluate potential effect measure modification.
Further, if the estimated disease risk score is strongly correlated with exposure status, the biases found by Pike and colleagues34
to be associated with stratification by the disease risk score estimated by the approach of Miettinen also apply to disease risk scores estimated in the unexposed.26
These concerns have probably contributed to the relatively infrequent use of disease risk scores in pharmacoepidemiology.35
However, if exposed and unexposed subjects differ substantially on important determinants of disease risk, such that the shared support of risk factor distributions is limited, then valid comparison of treatments in an observational setting becomes less feasible,36,37
and stratification on either a risk or propensity score is a useful way to identify such non-overlap. Rather than a limitation, the ready ability to identify the kinds of subjects who almost always receive one specific treatment, and who thus should probably not be included in an analysis of comparative effectiveness, is a strength of both propensity score and disease risk score methods in pharmacoepidemiology.35,36,38
The disease risk score can also be estimated with data from a time period prior to the study period, or from a separate population. However, one difficulty with estimation in a separate population is that covariate assessments may differ from those in the study population. In the context of evaluation of a new therapy, the time period before its introduction in the target study population may be useful. The reasoning behind this approach is that in times of evolving therapies, the disease risk in the population may be more stable than the propensity score. We illustrate this approach in the examples that follow.