|Home | About | Journals | Submit | Contact Us | Français|
To examine the effect of correcting coronary heart disease (CHD) risk factors for long-term within-person variation on CHD risk.
Using 5533 men and 7301 women from the Atherosclerosis Risk in Communities (ARIC) Study, we compared models incorporating risk factors measured at a single visit and models incorporating additional measurements for systolic blood pressure, total cholesterol and high-density lipoprotein cholesterol taken 3 years prior to baseline.
The largest change away from null was seen for systolic blood pressure: Hazard ratio (HR) 1.38 to 1.69 (+81%) in women and HR 1.26 to 1.41 (+56%) in men. Hazard ratios also decreased for age (−32% in women, −9% in men), race (−67% in women), diabetes (−13% in men and women), and medication use for hypertension (−27% in women, −26% in men) and cholesterol (−97% in women, HR 1.06 to 0.93 in men). The area under the ROC curve did not improve significantly in men or women, while reclassification was only significant in women (NRI 5.4%, p = 0.016).
Modeling long-term variation in CHD risk factors had a substantial impact on HR estimates, with new effect estimates further from the null for some risk factors and closer for others including age and medication use, but only improved risk classification in women.
Models of coronary heart disease (CHD) risk drive both public health policy decisions and individual treatment recommendations. However, CHD risk prediction usually relies on measurement of risk factors at a single point in time. Some risk factors are well described with one baseline measure, but others exhibit long term variation - a combination of instrument measurement error, physiologic short-term variability, and long term changes. Prediction and relative risks are usually estimated over years of follow-up, during which risk factor levels vary. These variations can affect the observed relationship of all risk factors to CHD, those with and without variation, and can affect both individual level prediction and population level estimates of relative risk (as well as odds ratios and relative hazards). Strachan and Rose noted that variation leads to underestimation of the potential benefits of both a population shift, for example to lower blood pressure, and a high risk reduction strategy, partially through an underestimation of the population attributable risk.(1) Incorrect assumptions about the effectiveness of screening can also result from the effect of variation on identification of high risk people.
Previous methods for accounting for the effect of variation in risk factors adjust the relative risk estimates directly and do not allow for adjustment to the absolute individual risk.(2–4) Consequently, these methods also do not allow empirical assessment of the potential improvement in prediction of disease risk obtained by accounting for variation, and the effect of accounting for variation on prediction is unknown. To address the question of whether accounting for risk factor variation has utility in prediction of disease risk, our study outlines a framework which allows for adjustment for correlations between risk factors in a measurement error model, in which measurements at different times are considered repeated measures of the long term average. This method is then applied to a United States community based sample of middle-aged black and white men and women who were measured at baseline and three years prior. We used regression calibration to obtain direct estimates of the long term average systolic blood pressure, total cholesterol and HDL cholesterol and examined the effect of using those values rather than a single measurement on relative hazards for CHD for both the measures with variation and those only measured at baseline, including race. We also assessed the effect of correction for long-term variation on prediction. This investigation provides one of the largest systematic implementations of the impact of long term variation in multiple risk factors on both the relative risk of CHD and its prediction.
The Atherosclerosis Risk in Communities (ARIC) Study is an ongoing prospective study that examines clinical and subclinical atherosclerotic diseases in a cohort of 15,792 persons, aged 45–64 years at baseline examination, selected from four U.S. communities. The sampling procedure and methods used in ARIC have been described in detail elsewhere.(5) The initial visit was followed by a visit three years later. For this study, the second visit was considered the baseline for single measurements of risk factors and for follow-up and information from the first visit (three years prior) was only used to estimate long-term variation. We excluded participants who reported ethnicity other than African-American or Caucasian (n=48), who reported prevalent CHD at the first visit or incident CHD prior to the second visit (n=1209), or had missing information on the risk factors at either visit (n=1701). After these exclusions, there were 12,834 participants for the present analysis (7301 women and 5533 men).
Information on age, gender, and race were obtained from home and clinic interviews conducted at the first visit. Smoking status was obtained from interview at each visit, as was current medication use.(6) Blood was collected at each visit, and total and HDL cholesterol testing was centralized.(7) Diabetes was defined at each visit as the presence of any one of the following: fasting glucose ≥ 126 mg/dL, non-fasting glucose ≥ 200 mg/dL, current use of diabetic medication, or reported physician diagnosis. Blood pressure was measured three times at each visit and the average of the second and third measurement was used in the analysis.(8)
CHD incidence was obtained through annual surveillance of patients, hospital records and death certificates through 2001. The abstraction and adjudication of these records has been previously described.(6, 9, 10) An event was defined as a definite or probable hospitalized myocardial infarction or a definite CHD death. Follow-up time was considered to start at the baseline (second) visit for all analyses.
Long-term variation was adjusted for in three continuous risk factors: HDL cholesterol, total cholesterol, and systolic blood pressure. Our method of adjustment is based on the estimation of a long-term average value, and dichotomous measures, such as current smoking status or diabetes diagnosis, do not fit as well into this framework. While a long term average could be estimated if desired, for this study only the baseline values of smoking status, diabetes diagnosis, and medication use were used. Age and race were also assumed to be well-characterized by a single measurement and the baseline values were included in all models. All models were stratified by gender.
We performed a series of analyses to explore the effect of different adjustment strategies. This first analysis was a Cox proportional hazards model in which baseline values alone were used for all risk factors with no adjustment for long-term variation. The second analysis used the mean of the baseline values and the visit three years prior for the continuous variables in a Cox proportional hazards model, adjusting for some variation but not using the covariance between risk factors. The third analysis, regression calibration as outlined by Carroll, Ruppert and Stefanski (11), used all the available relationships between all of the risk factors to generate an adjusted estimate for each continuous risk factor. Conceptually, this process estimates the actual desired variable, eg long term average blood pressure, using a weighted average of the best guess for an individual (the individual mean) and the mean for similar people (people with similar levels of the other risk factors), with the weight for the best guess for the individual being a function of the reliability of the measurement. The adjusted estimate was then used in a Cox proportional hazards model to obtain the prediction and relative hazard estimates. Substantial methodological and applied research has been dedicated in recent years to survival analysis with covariates subject to measurement error. (12–14) These analyses were implemented using R version 2.6.
We also conducted analyses using two available software packages: the method outlined by Rosner and colleagues (4) and available from Spiegleman and colleagues (15) in the relibpls macro in SAS and the rcal procedure in Stata, developed by Hardin et al (16). The relibpls procedure uses the variability in the repeated measures to directly adjust the effect estimates. The rcal procedure uses the same regression calibration structure as the proposed method, but does not directly generate the individual values for the variables with variation. The rcal procedure currently cannot run Cox proportional hazards models, so a Poisson model was used instead. For the mean and regression calibration models, percent change in excess relative hazard (RH) from the baseline model was also calculated as 100%*[(RHcorrected-1) - (RHbaseline-1)]/(RHbaseline-1) to show whether the change in the model strengthened or weakened the effect.
To compare predictive ability, we examined the AUC for the first three models. We were unable to calculate the AUC for the final two models because the corrected estimates of relative risk do not have corresponding corrected risk factor values. The AUC was generated using the Kaplan-Meier-like method of Chambless and Diao.(17) We also generated 10-year predicted CHD event risk for each participant using the baseline hazard function from each model and the individual covariates. Participants were grouped into deciles of 10-year risk for each model and observed event rates in each decile were calculated to assess calibration using the Hosmer-Lemeshow test. (18)
Risk reclassification (19, 20) was assessed by categorizing the predicted 10-year risk for each model into categories of less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or higher. We then compared the assigned categories for a pair of models and calculated the Net Reclassification Improvement (NRI), (21) which compares the shifts in reclassified categories by observed outcome, and the Integrated Discrimination Improvement (IDI), (21) which directly compares the average difference in predicted risk.
Table 1 summarizes the distribution of the key risk factors for the baseline visit and the distribution of the previous visit values. There was fluctuation in smoking status and diabetes categorization. There were current smokers at baseline who did not report smoking at the previous visit (2% men, 1% women) and people who did not report smoking at baseline but were current smokers at the previous visit (4% men and women). There were also new cases of diabetes at the baseline visit compared to three years earlier (6% men, 5% women) and people who were not considered diabetic at baseline but were at the previous visit (1% men and women). Table 1 also shows the correlation between the baseline and previous visit for systolic blood pressure, total and HDL cholesterol which are the focus of correction for long term variation.
Table 2 displays the correlations between the mean values of the time-varying risk factors and the baseline values of the factors modeled without variation for men and women. Pearson correlations are shown when both factors are continuous and point biserial correlations are shown when one is dichotomous. These relationships are used in the regression calibration and explain some of the changes in the relative hazard estimation. Of note, age is correlated with blood pressure and HDL cholesterol in men and only blood pressure in women, while African-American race is correlated with higher blood pressure in men and women and additionally with higher total cholesterol in women. The mean values of blood pressure are also more correlated with the mean values of HDL and total cholesterol in women than in men.
Tables 3 and and44 show the results of the multivariate analyses in men and women, respectively. Figure 1 further highlights the change in relative hazard between the baseline and regression calibration models. As shown, the effect of adjustment is strongest using the regression calibrated approach, but using the mean of the two visits changes relative hazards in the same direction. While the specific estimates of the models vary slightly, the relationship of the Rosner and rcal results to the estimates of effect in the baseline model is consistent. Comparing the baseline model to the regression calibrated model, we observe an increase in the relative hazard of total cholesterol of 51% in men and 74% in women and an increase in the relative hazard of systolic blood pressure of 56% in men and 81% in women. The relative hazard for HDL changed from 0.62 to 0.53 in men, a 26% increase in effect, and from 0.77 to 0.68 in women, a 43% increase in effect. We also observed decreases in the effect of the factors modeled without variation in the regression calibration model with the exception of smoking. The relative hazard for age decreased 9% in men and 32% in women, the relative hazard for race decreased 67% in women and increased 5% in men, and the relative hazard for diabetes decreased 13% in men and women. The relative hazard for hypertension medication use decreased 26% in men and 27% in women, while the relative hazard for cholesterol medication use decreased 97% in women and went from 1.06 to 0.93 in men. The relative hazard estimates for smoking remained relatively unchanged, increasing slightly in men (1%) and women (7%).
As shown in Table 5, the AUC increased from 0.701 for the baseline model to 0.704 for the mean model in men and from 0.780 to 0.785 in women. However, no further increase was seen for the regression calibration model, suggesting that the slight increase was due to the additional information from the previous visit rather than the specific model. Bootstrapped 95% confidence intervals for the difference between the baseline and regression calibrated models were (−0.020, 0.034) for men and (−0.029, 0.041) for women, showing no significant improvement in the AUC using the regression calibrated model. All models were calibrated and no improvement in reclassification measures was seen using multiple measurements in the men. In the women, however, reclassification was improved by using a combined measurement over the baseline value (NRI 6.4%, p = 0.008)
This study examined the effect of correction for long-term variation in multiple risk factors on CHD risk estimation and prediction. Correcting for long term variation has a substantial effect onthe relative hazard estimates, strengthening the relative hazards for systolic blood pressure, total and HDL cholesterol, with little change for smoking, and weakening of the relative hazards of age, medication use, and, in women, race. The regression calibration method generated an estimate of the value of the risk factors corrected for long-term variation, which allowed us to also examine the effect of correction on prediction. There was no significant increase in the AUC in the mean model compared to the baseline model, and no further improvement in the regression calibration model. However, both the mean and the regression calibration models improved risk classification in women compared to the baseline model.
Some of the changes in hazard ratios due to adjustment for long term variation are suggested by underlying risk factor relationships. For example, cholesterol and hypertension medication use at baseline are likely to be related to cholesterol and blood pressure values 3-years prior. Likewise the decrease in the hazard ratios for age in women might be related to the stronger relationship between age and blood pressure in women than men. The lack of change in the hazard ratio of smoking could be due to a lack of relationship between smoking status and blood pressure and cholesterol levels.
This work builds on previously published findings on CHD risk from the ARIC cohort (22). The increases seen in our estimates of relative effect are slightly higher than those shown by single risk factor adjustment. MacMahon et al (23) found a 60% increase in the relative risk for blood pressure, while Law et al (24) and Verschuren et al (25) found increases in the relative risk for of total cholesterol of around 40%.
The results of multivariable adjustment, on the other hand, will depend on which risk factors are included in the model, though our results show similar direction to those seen in other studies. Rosner et al (4) found an increase in the effects of cholesterol, glucose and blood pressure, when variation in those variables was included, and a decrease in the effect of smoking and BMI and age. Iribarren et al (26) showed increases in the hazard ratios of serum cholesterol, blood pressure, and dietary cholesterol, when variation was included, and mixed effects in smoking, and decreases in the hazard ratios of alcohol consumption and abstinence and body mass index.
Emberson et al (27) used a model in which diastolic blood pressure and serum total cholesterol were assumed to vary, while age, history of CHD, and smoking status were not and found increases in all hazard ratios. To aid in the comparison, we provided the relative hazards using the same variables but generated using the Rosner macro, which were very similar to those obtained using our regression calibration method. This suggests that differences in the results are not driven by the choice of method.
Previous studies have not examined the effect of correction for long term variation on prediction. Our results show no significant improvement in discrimination resulting from the additional information provided by the previous visit. This lack of improvement did not vary across different methods of accounting for long term variation. The improvement in classification seen in women but not in men may be due to the stronger correlations between the two measurements in women, making the long term average a better estimate of the effect in women. This finding would need to be replicated in additional populations. Additionally, the utility of our method in adding predictive ability for other diseases, or if expanded by using more than two measurements to include an estimate of trajectory rather than averages, remains untested. It is clear, however, that using a corrected model does not decrease predictive ability of the model. One potential advantage of correction for variation on prediction worthy of further study is that it allows for the risk model not to depend on the variability in a given population. Correction for variation may allow for more similarity in models and calibration across populations.
This study is limited by having only two visits, three years apart, for each variable, and did not include continuous measures of smoking and glycemic control. With more frequent measurements, which may become increasingly available through electronic medical records, the method could be extended to examine trajectories of change rather than a single long-term value. Including continuous measures of glucose control and smoking would have allowed improved understanding of the relationships between risk factors. Additionally, we were also unable to separate the observed long-term variation into measurement imprecision, short-term variability and long-term changes. However, decomposition would not impact the final results of the present analysis. Also, our correction method is a first-order correction and does not take into account the variance of the underlying estimate of the long-term mean. We did check for a period effect and for a relationship between the variability and age at baseline and found none. Variation in the mean time between visits (mean 2.9 years, SD 0.2) in men and women led to some variability between the ages at the two visits (correlation of 0.998 in men and 0.997 in women).. We were also limited by the missing values in the data. We chose to use a listwise deletion approach rather than use the other variables to both estimate the missing data and to derive the correlation structure for the regression calibration. However, our approach does introduce potential biases in the results if the missing values are not randomly distributed.
The study does have substantial strengths. The time period of measurements and follow-up is well suited to answer the question of previous variation on future risk prediction versus relative hazard estimation. We also were able to correct for variation in multiple risk factors simultaneously as well as see the effects of correction on the remaining risk factors. Finally, our method provided an actual estimate of the underlying risk factor value and allowed us to examine prediction.
We found the relative risks of CHD associated with higher blood pressure, total and HDL cholesterol are substantially closer to the null while the relative risks for age, race (in women), and diabetes are further from the null when models ignore long term variation. In contrast, we found that adding one visit prior to baseline only improved CHD risk classification in women. In addition, the methods we present allow estimation of the underlying risk factor values and assessment of any change in prediction. Our results suggest that correction for long-term variation using this method of regression calibration is important in understanding the relative and predictive effects of CHD risk factors.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.