Sensitive general cardiometabolic risk assessment tools of modifiable risk factors would be helpful and practical in a range of primary prevention interventions or for preventive health maintenance.
To develop and validate a cumulative general cardiometabolic risk score that focuses on non–self-reported modifiable risk factors such as glycosylated hemoglobin (HbA1c) and BMI so as to be sensitive to small changes across a span of major modifiable risk factors, which may not individually cross clinical cut off points for risk categories.
We prospectively followed 2,359 cardiovascular disease (CVD)-free subjects from the Framingham offspring cohort over a 14–year follow-up. Baseline (fifth offspring examination cycle) included HbA1c and cholesterol measurements. Gender–specific Cox proportional hazards models were considered to evaluate the effects of non–self-reported modifiable risk factors (blood pressure, total cholesterol, high–density lipoprotein cholesterol, smoking, BMI, and HbA1c) on general CVD risk. We constructed 10–year general cardiometabolic risk score functions and evaluated its predictive performance in 2012–2013.
HbA1c was significantly related to general CVD risk. The proposed cardiometabolic general CVD risk model showed good predictive performance as determined by cross-validated discrimination (male C-index=0.703, 95% CI=0.668, 0.734; female C-index=0.762, 95% CI=0.726, 0.801) and calibration (lack-of-fit χ2=9.05 [p=0.338] and 12.54 [p=0.128] for men and women, respectively).
This study presents a risk factor algorithm that provides a convenient and informative way to quantify cardiometabolic risk based on modifiable risk factors that can motivate an individual’s commitment to prevention and intervention.
It is unclear to what extent the incremental predictive performance of a novel biomarker is impacted by the method used to control for standard predictors. We investigated whether adding a biomarker to a model with a published risk score overestimates its incremental performance as compared to adding it to a multivariable model with individual predictors (or a composite risk score estimated from the sample of interest), and to a null model. We used 1000 simulated datasets (with a range of risk factor distributions and event rates) to compare these methods, using the continuous Net Reclassification Index (NRI), the Integrated Discrimination Index (IDI), and change in the C-statistic as discrimination metrics. The new biomarker was added to a: null model; model including a published risk score; model including a composite risk score estimated from the sample of interest; and multivariable model with individual predictors. We observed a gradient in the incremental performance of the biomarker, with the null model resulting in the highest predictive performance of the biomarker and the model using individual predictors resulting in the lowest (mean increases in C-statistic between models without and with the biomarker: 0.261, 0.085, 0.030, and 0.031; NRI: 0.767, 0.621, 0.513, and 0.530; IDI: 0.153, 0.093, 0.053 and 0.057, respectively). These findings were supported by Framingham Study data predicting atrial fibrillation using novel biomarkers. We recommend that authors report the effect of a new biomarker after controlling for standard predictors modeled as individual variables.
biomarkers; model discrimination; risk model; risk prediction
The Net Reclassification Improvement (NRI) has become a popular metric for evaluating improvement in disease prediction models through the past years. The concept is relatively straightforward but usage and interpretation has been different across studies. While no thresholds exist for evaluating the degree of improvement, many studies have relied solely on the significance of the NRI estimate. However, recent studies recommend that statistical testing with the NRI should be avoided. We propose using confidence ellipses around the estimated values of event and non-event NRIs which might provide the best measure of variability around the point estimates. Our developments are illustrated using practical examples from EPIC-Potsdam study.
Risk assessment; Risk model; Model comparison; Reclassification; Confidence intervals
To assess the prognostic value of 12-months N-Terminal Pro-Brain Natriuretic Peptide (NT-proBNP) levels on adverse cardiovascular events in patients with stable coronary heart disease.
NT-proBNP concentrations were measured at baseline and at 12-months follow-up in participants of cardiac rehabilitation (median follow-up 8.96 years). Cox-proportional hazards models evaluated the prognostic value of log-transformed NT-proBNP levels, and of 12-months NT-proBNP relative changes on adverse cardiovascular events adjusting for established risk factors measured at baseline.
Among 798 participants (84.7% men, mean age 59 years) there were 114 adverse cardiovascular events. 12-months NT-proBNP levels were higher than baseline levels in 60 patients (7.5%) and numerically more strongly associated with the outcome in multivariable analysis (HR 1.65 [95% CI 1.33–2.05] vs. HR 1.41 [95% CI 1.12–1.78], with a net reclassification improvement (NRI) of 0.098 [95% CI 0.002–0.194] compared to NRI of 0.047 [95% CI −0.0004–0.133] for baseline NT-proBNP levels. A 12-month 10% increment of NT-proBNP was associated with a HR of 1.35 [95% CI 1.12–1.63] for the onset of an adverse cardiovascular event. Subjects with a 12-month increment of NT-proBNP had a HR of 2.56 [95% CI 1.10–5.95] compared to those with the highest 12-months reduction.
Twelve-months NT-proBNP levels after an acute cardiovascular event are strongly associated with a subsequent event and may provide numerically better reclassification of patients at risk for an adverse cardiovascular event compared to NT-proBNP baseline levels after adjustment for established risk factors.
The discrimination of a risk prediction model measures that model's ability to distinguish between subjects with and without events. The area under the receiver operating characteristic curve (AUC) is a popular measure of discrimination. However, the AUC has recently been criticized for its insensitivity in model comparisons in which the baseline model has performed well. Thus, 2 other measures have been proposed to capture improvement in discrimination for nested models: the integrated discrimination improvement and the continuous net reclassification improvement. In the present study, the authors use mathematical relations and numerical simulations to quantify the improvement in discrimination offered by candidate markers of different strengths as measured by their effect sizes. They demonstrate that the increase in the AUC depends on the strength of the baseline model, which is true to a lesser degree for the integrated discrimination improvement. On the other hand, the continuous net reclassification improvement depends only on the effect size of the candidate variable and its correlation with other predictors. These measures are illustrated using the Framingham model for incident atrial fibrillation. The authors conclude that the increase in the AUC, integrated discrimination improvement, and net reclassification improvement offer complementary information and thus recommend reporting all 3 alongside measures characterizing the performance of the final model.
area under curve; biomarkers; discrimination; risk assessment; risk factors
Higher left ventricular (LV) mass, wall thickness and internal dimension
are associated with increased heart failure (HF) risk. Whether different LV
hypertrophy patterns vary with respect to rates and types of HF incidence is
unclear. We classified 4768 Framingham Heart Study participants (mean age 50
years; 56% women) into 4 mutually exclusive LV hypertrophy pattern
groups (normal, concentric remodeling, concentric hypertrophy, eccentric
hypertrophy) using American Society of Echocardiography recommended thresholds
of echocardiographic LV mass/body surface area and relative wall thickness, and
related them to HF incidence. We evaluated if risk for HF types (HF with reduced
[<45%; HFREF] versus preserved
[≥45%; HFPEF] ejection fraction) varied by
hypertrophy pattern. On follow-up (mean 21 years), 458 participants
(9.6%; 250 women) developed new-onset HF. The age-and-sex-adjusted
20-year HF incidence rose from 6.96% in normal LV group to
8.67%, 13.38% and 15.27% in the concentric remodeling,
concentric hypertrophy and eccentric hypertrophy groups, respectively. After
adjustment for co-morbidities and incident myocardial infarction, LV hypertrophy
patterns were associated with higher HF incidence relative to normal LV
(p=0.0002); eccentric hypertrophy carried the greatest risk (hazards
ratio [HR] 1.89, 95% confidence interval
[CI] 1.41-2.54), followed by concentric hypertrophy (HR
[CI] 1.40 [1.04-1.87]). Participants with
eccentric hypertrophy had a higher propensity for HFREF (HR 2.23; CI 1.48-3.37,
whereas those with concentric hypertrophy were more prone to HFPEF (HR 1.66; CI
1.09-2.51). In conclusion, in our large community-based sample, HF risk varied
by LV hypertrophy pattern, with eccentric and concentric hypertrophy
predisposing to HFREF and HFPEF, respectively.
Concentric hypertrophy; eccentric hypertrophy; left ventricular hypertrophy; heart failure; risk
Coronary computed tomographic (CT) angiography is a noninvasive anatomic test for diagnosis of coronary stenosis that does not determine whether a stenosis causes ischemia. In contrast, fractional flow reserve (FFR) is a physiologic measure of coronary stenosis expressing the amount of coronary flow still attainable despite the presence of a stenosis, but it requires an invasive procedure. Noninvasive FFR computed from CT (FFRCT) is a novel method for determining the physiologic significance of coronary artery disease (CAD), but its ability to identify ischemia has not been adequately examined to date.
To assess the diagnostic performance of FFRCT plus CT for diagnosis of hemodynamically significant coronary stenosis.
Design, Setting, and Patients
Multicenter diagnostic performance study involving 252 stable patients with suspected or known CAD from 17 centers in 5 countries who underwent CT, invasive coronary angiography (ICA), FFR, and FFRCT between October 2010 and October 2011. Computed tomography, ICA, FFR, and FFRCT were interpreted in blinded fashion by independent core laboratories. Accuracy of FFRCT plus CT for diagnosis of ischemia was compared with an invasive FFR reference standard. Ischemia was defined by an FFR or FFRCT of 0.80 or less, while anatomically obstructive CAD was defined by a stenosis of 50% or larger on CT and ICA.
Main Outcome Measures
The primary study outcome assessed whether FFRCT plus CT could improve the per-patient diagnostic accuracy such that the lower boundary of the 1-sided 95% confidence interval of this estimate exceeded 70%.
Among study participants, 137 (54.4%) had an abnormal FFR determined by ICA. On a per-patient basis, diagnostic accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of FFRCT plus CT were 73% (95% CI, 67%–78%), 90% (95% CI, 84%–95%), 54% (95% CI, 46%–83%), 67% (95% CI, 60%–74%), and 84% (95% CI, 74%–90%), respectively. Compared with obstructive CAD diagnosed by CT alone (area under the receiver operating characteristic curve [AUC], 0.68; 95% CI, 0.62–0.74), FFRCT was associated with improved discrimination (AUC, 0.81; 95% CI, 0.75–0.86; P<.001).
Although the study did not achieve its prespecified primary outcome goal for the level of per-patient diagnostic accuracy, use of noninvasive FFRCT plus CT among stable patients with suspected or known CAD was associated with improved diagnostic accuracy and discrimination vs CT alone for the diagnosis of hemodynamically significant CAD when FFR determined at the time of ICA was the reference standard.
To determine change in the prevalence of functional limitations and physical disability in community-dwelling elders across three decades.
We studied original participants of the Framingham Study, aged 79 to 88 years, at exam 15 (1977–1979, 177 women, 103 men), exam 20 (1988–1990, 159 women, 98 men) and exam 25 (1997 to 1999, 174 women, 119 men). Self-reported 1) functional limitation defined using the Nagi scale and 2) physical disability defined using the Rosow-Breslau and Katz scales.
Functional limitations declined across examinations from 74.6% to 60.5% to 37.9% (p< 0.001) in women and 54.2%, 37.8%, and 27.8% (p<0.001) in men. Physical disability declined from 74.5% to 48.5% to 34.6% (p< 0.001) in women and 42.3% to 33.3% to 22.8% (p=0.009) in men. Women had a greater decline in disability than men (p=0.03). In women, improvements in functional limitations (p=0.05) were greater from exam 20 to 25 whereas for physical disability (p=0.02) improvements were greater from exam 15 to 20. Improvements in function were constant across the three examinations in men.
Among community-dwelling elders the prevalence of functional limitations and physical disability declined significantly from the 1970s to the 1990s.
functional limitations; physical disability; trends; elders
To determine whether biomarkers of myocardial stress and fibrosis improve prediction of mode of death in patients with chronic heart failure.
The two most common modes of death in patients with chronic heart failure are pump failure and sudden cardiac death. Prediction of mode of death may facilitate treatment decisions. The relationship between NT-proBNP, galectin-3, and ST2, biomarkers that reflect different pathogenic pathways in heart failure (myocardial stress and fibrosis), and mode of death is unknown.
HF-ACTION was a randomized controlled trial of exercise training vs. usual care in patients with chronic heart failure due to left ventricular systolic dysfunction (LVEF<35%). An independent clinical events committee prospectively adjudicated mode of death. NT-proBNP, galectin-3, and ST2 levels were assessed at baseline in 813 subjects. Associations between biomarkers and mode of death were assessed using cause-specific Cox-proportional hazards modeling, and interaction testing was used to measure differential association between biomarkers and pump failure versus sudden cardiac death. Discrimination and risk reclassification metrics were used to assess the added value of galectin-3 and ST2 in predicting mode of death risk beyond a clinical model that included NT-proBNP.
After a median follow up of 2.5 years, there were 155 deaths: 49 from pump failure 42 from sudden cardiac death, and 64 from other causes. Elevations in all biomarkers were associated with increased risk of both pump failure and sudden cardiac death in both adjusted and unadjusted analyses. In each case, increases in the biomarker had a stronger association with pump failure than sudden cardiac death but this relationship was attenuated after adjustment for clinical risk factors. Clinical variables along with NT-proBNP levels were stronger predictors of pump failure (C statistic: 0.87) than sudden cardiac death (C statistic: 0.73). Addition of ST2 and galectin-3 led to improved net risk classification of 11% for sudden cardiac death, but not pump failure.
Clinical predictors along with NT-proBNP levels were strong predictors of pump failure risk, with insignificant incremental contributions of ST2 and galectin-3. Predictability of sudden cardiac death risk was less robust and enhanced by information provided by novel biomarkers.
heart failure; biomarker; prognosis; mode of death
In this paper we investigate how the correlation structure of independent
variables affects the discrimination of risk prediction model. Using multivariate normal
data and binary outcome we prove that zero correlation among predictors is often
detrimental for discrimination in a risk prediction model and negatively correlated
predictors with positive effect sizes are beneficial. A very high multiple R-squared from
regressing the new predictor on the old ones can also be beneficial. As a practical guide
to new variable selection, we recommend to select predictors that have negative
correlation with the risk score based on the existing variables. This step is easy to
implement even when the number of new predictors is large. Our results are illustrated
using real-life Framingham data suggesting that the conclusions hold outside of normality.
The findings presented in this paper might be useful for preliminary selection of
potentially important predictors, especially is situations where the number of predictors
AUC; discrimination; risk prediction model; correlation; linear discriminant analysis; logistic regression
Independent data monitoring committees (IDMCs) were introduced to monitor patient safety and study conduct in randomized clinical trials (RCTs), but certain challenges regarding the utilization of IDMCs have developed. First, the roles and responsibilities of IDMCs are expanding, perhaps due to increasing trial complexity and heterogeneity regarding medical, ethical, legal, regulatory, and financial issues. Second, no standard for IDMC operating procedures exists, and there is uncertainty about who should determine standards and whether standards should vary with trial size and design. Third, considerable variability in communication pathways exist across IDMC interfaces with regulatory agencies, academic coordinating centers, and sponsors. Finally, there has been a substantial increase in the number of RCTs using IDMCs, yet there is no set of qualifications to help guide the training and development of the next generation of IDMC members. Recently, an expert panel of representatives from government, industry, and academia assembled at the Duke Clinical Research Institute to address these challenges and to develop recommendations for the future utilization of IDMCs in RCTs.
The association of familial as compared to genetic factors in the current obesogenic environment, compared to earlier, leaner time periods, is uncertain.
Design and Methods
Participants from the Framingham Heart Study were classified according to parental obesity status in the Original, Offspring, and Third Generation cohorts; mean BMI levels were estimated and we compared the association of parental history across generations. Finally, a genetic risk score comprised of 32 well-replicated single nucleotide polymorphisms for BMI was examined in association with BMI levels in 1948, 1971, and 2002.
BMI was 1.49 kg/m2 higher per each affected parent among the Offspring, and increased to 2.09 kg/m2 higher among the Third Generation participants (p-value for the cohort comparison=0.007). Parental history of obesity was associated with increased weight gain (p<0.0001) and incident obesity (p=0.009). Despite a stronger association of parental obesity with offspring BMI in more contemporary time periods, we observed no change in the effect size of a BMI genetic risk score from 1948 to 2002 (p=0.11 for test of trend across the time periods).
The association of parental obesity has become stronger in more contemporary time period, whereas the association of a BMI genetic risk score has not changed.
obesity; epidemiology; weight change; family history; Framingham Heart Study
Atrial fibrillation (AF) is a strong risk factor for heart failure (HF); HF onset in patients with AF is associated with increased morbidity and mortality. Risk factors that predict HF in individuals with AF in the community are not well established.
Methods and results
We examined clinical variables related to the 10-year incidence of HF in 725 individuals (mean 73.3 years, 45% women) with documented AF in the Framingham Heart Study. Event rates for incident HF (n = 161, 48% in women) were comparable in women (4.30 per 100 person-years) and men (3.34 per 100 person-years). Age, body mass index, ECG LV hypertrophy, diabetes, significant murmur, and history of myocardial infarction were positively associated with incident HF in multivariable models (C-statistic 0.71; 95% confidence interval 0.67–0.75). We developed a risk algorithm for estimating absolute risk of HF in AF patients with good model fit and calibration (adjusted calibration χ2 statistic 7.29; Pχ2 = 0.61). Applying the algorithm, 47.6% of HF events occurred in the top tertile in men compared with 13.1% in the bottom tertile, and 58.4% in women in the upper tertile compared with 18.2% in the lowest category. For HF type, women had a non-significantly higher incidence of HF with preserved EF compared with men.
We describe advancing age, LV hypertrophy, body mass index, diabetes, significant heart murmur, and history of myocardial infarction as clinical predictors of incident HF in individuals with AF. A risk algorithm may help identify individuals with AF at high risk of developing HF.
Atrial fibrillation; Risk score; Epidemiology; Heart failure
For the evaluation and comparison of markers and risk prediction models, various novel measures have recently been introduced as alternatives to the commonly used difference in the area under the ROC curve (ΔAUC). The Net Reclassification Improvement (NRI) is increasingly popular to compare predictions with one or more risk thresholds, but decision-analytic approaches have also been proposed.
We aimed to identify the mathematical relationships between novel performance measures for the situation that a single risk threshold T is used to classify patients as having the outcome or not.
We considered the NRI and three utility-based measures that take misclassification costs into account: difference in Net Benefit (ΔNB), difference in Relative Utility (ΔRU), and weighted NRI (wNRI). We illustrate the behavior of these measures in 1938 women suspect of ovarian cancer (prevalence 28%).
The three utility-based measures appear transformations of each other, and hence always lead to consistent conclusions. On the other hand, conclusions may differ when using the standard NRI, depending on the adopted risk threshold T, prevalence P and the obtained differences in sensitivity and specificity of the two models that are compared. In the case study, adding the CA-125 tumor marker to a baseline set of covariates yielded a negative NRI yet a positive value for the utility-based measures.
The decision-analytic measures are each appropriate to indicate the clinical usefulness of an added marker or compare prediction models, since these measures each reflect misclassification costs. This is of practical importance as these measures may thus adjust conclusions based on purely statistical measures. A range of risk thresholds should be considered in applying these measures.
The aim of the current study is to assess the mortality prediction accuracy of circulating cell-free DNA (CFD) level at admission measured by a new simplified method.
Materials and Methods
CFD levels were measured by a direct fluorescence assay in severe sepsis patients on intensive care unit (ICU) admission. In-hospital and/or twenty eight day all-cause mortality was the primary outcome.
Out of 108 patients with median APACHE II of 20, 32.4% have died in hospital/or at 28-day. CFD levels were higher in decedents: median 3469.0 vs. 1659 ng/ml, p<0.001. In multivariable model APACHE II score and CFD (quartiles) were significantly associated with the mortality: odds ratio of 1.05, p = 0.049 and 2.57, p<0.001 per quartile respectively. C-statistics for the models was 0.79 for CFD and 0.68 for APACHE II. Integrated discrimination improvement (IDI) analyses showed that CFD and CFD+APACHE II score models had better discriminatory ability than APACHE II score alone.
CFD level assessed by a new, simple fluorometric-assay is an accurate predictor of acute mortality among ICU patients with severe sepsis. Comparison of CFD to APACHE II score and Procalcitonin (PCT), suggests that CFD has the potential to improve clinical decision making.
Age trends in estradiol and estrone levels in men and how lifestyle factors, comorbid conditions, testosterone, and sex hormone–binding globulin affect these age trends remain poorly understood, and were examined in men of the Framingham Heart Study.
Estrone and estradiol concentrations were measured in morning fasting samples using liquid chromatography tandem mass spectrometry in men of Framingham Offspring Generation. Free estradiol was calculated using a law of mass action equation.
There were 1,461 eligible men (mean age [±SD] 61.1±9.5 years and body mass index [BMI] 28.8±4.5kg/m2). Total estradiol and estrone were positively associated with age, but free estradiol was negatively associated with age. Age-related increase in total estrone was greater than that in total estradiol. Estrone was positively associated with smoking, BMI, and testosterone, and total and free estradiol with diabetes, BMI, testosterone, and comorbid conditions; additionally, free estradiol was associated negatively with smoking. Collectively, age, BMI, testosterone, and other health and behavioral factors explained only 18% of variance in estradiol, and 9% of variance in estrone levels. Men in the highest quintile of estrone levels had significantly higher age and BMI, and a higher prevalence of smoking, diabetes, and cardiovascular disease than others, whereas those in the highest quintile of estradiol had higher BMI than others.
Total estrone and estradiol levels in men, measured using liquid chromatography tandem mass spectrometry, revealed significant age-related increases that were only partially accounted for by cross-sectional differences in BMI, diabetes status, and other comorbidities and health behaviors. Longitudinal studies are needed to confirm these findings.
Age trends; Estrogen levels in men; LC-MS/MS; Age-related changes in estrone and estradiol; Determinants of estrogen levels in men.
Many studies of diabetes have examined risk factors at the time of diabetes diagnosis instead of considering the lifetime burden of adverse risk factor levels. We examined the 30-year cardiovascular disease (CVD) risk factor burden that participants have up to the time of diabetes diagnosis.
RESEARCH DESIGN AND METHODS
Among participants free of CVD, incident diabetes cases (fasting plasma glucose ≥126 mg/dL or treatment) occurring at examinations 2 through 8 (1979–2008) of the Framingham Heart Study Offspring cohort were age- and sex-matched 1:2 to controls. CVD risk factors (hypertension, high LDL cholesterol, low HDL cholesterol, high triglycerides, obesity) were measured at the time of diabetes diagnosis and at time points 10, 20, and 30 years prior. Conditional logistic regression was used to compare risk factor levels at each time point between diabetes cases and controls.
We identified 525 participants with new-onset diabetes who were matched to 1,049 controls (mean age, 60 years; 40% women). Compared with those without diabetes, individuals who eventually developed diabetes had higher levels of hypertension (odds ratio [OR], 2.2; P = 0.003), high LDL (OR, 1.5; P = 0.04), low HDL (OR, 2.1; P = 0.0001), high triglycerides (OR, 1.7; P = 0.04), and obesity (OR, 3.3; P < 0.0001) at time points 30 years before diabetes diagnosis. After further adjustment for BMI, the ORs for hypertension (OR, 1.9; P = 0.02) and low HDL (OR, 1.7; P = 0.01) remained statistically significant.
CVD risk factors are increased up to 30 years before diagnosis of diabetes. These findings highlight the importance of a life course approach to CVD risk factor identification among individuals at risk for diabetes.
The discovery and development of new biomarkers continues to be an exciting and promising field. Improvement of prediction of risk of developing disease is one of the key motivations in these pursuits. Appropriate statistical measures are necessary for drawing meaningful conclusions about the clinical usefulness of these new markers. In this review, we present several novel metrics proposed to serve this purpose. We use reclassification tables constructed based on clinically meaningful disease risk categories to discuss the concepts of calibration, risk separation, risk discrimination, and risk classification accuracy. We discuss the notion that the net reclassification improvement is a simple yet informative way to summarize information contained in risk reclassification tables. In the absence of meaningful risk categories, we suggest a ‘category-less’ version of the net reclassification improvement and integrated discrimination improvement as metrics to summarize the incremental value of new biomarkers. We also suggest that predictiveness curves be preferred to receiver-operating-characteristic curves as visual descriptors of a statistical model’s ability to separate predicted probabilities of disease events. Reporting of standard metrics, including measures of relative risk and the c statistic is still recommended. These concepts are illustrated with a risk prediction example using data from the Framingham Heart Study.
reclassification; risk prediction; NRI; IDI; calibration; discrimination
Hypertension is a risk factor for coronary artery disease. Recent genome-wide association studies have identified 30 genetic variants associated with higher blood pressure at genome-wide significance (p<5×10−8). If elevated blood pressure is a causative factor for coronary artery disease, these variants should also increase coronary artery disease risk. Analyzing genome-wide association data from 22,233 coronary artery disease cases and 64,762 controls, we observed in the Coronary artery disease Genome-Wide Replication And Meta-Analysis (CARDIoGRAM) consortium that 88% of these blood pressure-associated polymorphisms were likewise positively associated with coronary artery disease, i.e. they had an odds ratio >1 for coronary artery disease, a proportion much higher than expected by chance (p=4.10−5). The average relative coronary artery disease risk increase per each of the multiple blood pressure-raising alleles observed in the consortium was 3.0% for systolic blood pressure-associated polymorphisms (95% confidence interval, 1.8 to 4.3%) and 2.9% for diastolic blood pressure-associated polymorphisms (95% confidence interval, 1.7 to 4.1%). In sub-studies, individuals carrying most systolic blood pressure- and diastolic blood pressure-related risk alleles (top quintile of a genetic risk score distribution) had 70% (95% confidence interval, 50-94%) and 59% (95% confidence interval, 40-81%) higher odds of having coronary artery disease, respectively, as compared to individuals in the bottom quintile. In conclusion, most blood pressure-associated polymorphisms also confer an increased risk for coronary artery disease. These findings are consistent with a causal relationship of increasing blood pressure to coronary artery disease. Genetic variants primarily affecting blood pressure contribute to the genetic basis of coronary artery disease.
Blood pressure; polymorphism; genetics; coronary artery disease
The area under the receiver operating characteristic curve (AUC) is the most commonly reported measure of discrimination for prediction models with binary outcomes. However, recently it has been criticized for its inability to increase when important risk factors are added to a baseline model with good discrimination. This has led to the claim that the reliance on the AUC as a measure of discrimination may miss important improvements in clinical performance of risk prediction rules derived from a baseline model. In this paper we investigate this claim by relating the AUC to measures of clinical performance based on sensitivity and specificity under the assumption of multivariate normality. The behavior of the AUC is contrasted with that of discrimination slope. We show that unless rules with very good specificity are desired, the change in the AUC does an adequate job as a predictor of the change in measures of clinical performance. However, stronger or more numerous predictors are needed to achieve the same increment in the AUC for baseline models with good versus poor discrimination. When excellent specificity is desired, our results suggest that the discrimination slope might be a better measure of model improvement than AUC. The theoretical results are illustrated using a Framingham Heart Study example of a model for predicting the 10-year incidence of atrial fibrillation.
risk prediction; discrimination; AUC; IDI; Youden index; relative utility
A breast cancer risk prediction model for black women, developed from data in the Women’s Contraceptive and Reproductive Experiences (CARE) study, has been validated in women aged 50 years or older but not among younger women or for specific breast cancer subtypes.
We assessed calibration and discrimination of the CARE model in the Black Women’s Health Study (BWHS) with data from 45 942 women aged 30 to 69 years at baseline.
During a mean follow-up of 9.5 years, we identified 852 invasive breast cancers. The CARE model predicted 749.6 breast cancers, yielding an expected-to-observed (E/O) ratio of 0.88 (95% confidence interval [CI] = 0.82 to 0.94). The E/O ratio did not appreciably differ between women aged less than 50 years and those aged 50 years or older. The model underpredicted risk to the greatest degree among women aged 25 years or older at birth of first child (E/O = 0.71, 95% CI = 0.63 to 0.81); the model was well calibrated among women aged less than 25 years at birth of first child. The prevalence of later age at birth of first child was higher in the BWHS than in the CARE study, and breast cancer incidence was higher in the BWHS compared with national rates used in the CARE model. With respect to discriminatory accuracy, the concordance statistic was 0.57 (95% CI = 0.55 to 0.59) for breast cancer overall, 0.59 (95% CI = 0.57 to 0.61) for estrogen receptor (ER)-positive breast cancer, and 0.54 (95% CI = 0.50 to 0.57) for ER-negative breast cancer.
The CARE model underpredicted breast cancer risk in the BWHS, at least in part because of older age at first birth in this cohort, which led to higher breast cancer incidence rates. Our results suggest that inclusion of age at first birth may improve model performance. Discriminatory accuracy was modest and worse for ER-negative breast cancer.
Cardiovascular disease (CVD) is among the leading causes of death and disability worldwide. Since its beginning, the Framingham study has been a leader in identifying CVD risk factors. Clinical trials have demonstrated that when the modifiable risk factors are treated and corrected, the chances of CVD occurring can be reduced. The Framingham study also recognized that CVD risk factors are multifactorial and interact over time to produce CVD. In response, Framingham investigators developed the Framingham Risk Functions (also called Framingham Risk Scores) to evaluate the chance or likelihood of developing CVD in individuals. These functions are multivariate functions (algorithms) that combine the information in CVD risk factors such as sex, age, systolic blood pressure, total cholesterol, high-density lipoprotein cholesterol, smoking behavior, and diabetes status to produce an estimate (or risk) of developing CVD or a component of CVD (such as coronary heart disease, stroke, peripheral vascular disease, or heart failure) over a fixed time, for example, the next 10 years. These estimates of CVD risk are often major inputs in recommending drug treatments such as cholesterol-lowering drugs.
Screening for osteoporosis with bone mineral density (BMD) is
recommended for older adults. It is unclear whether repeating a BMD
screening test improves fracture risk assessment.
To determine whether changes in BMD after 4 years provide additional
information on fracture risk beyond baseline BMD and to quantify the change
in fracture risk classification after a second BMD measure.
DESIGN, SETTING, AND PARTICIPANTS
Population-based cohort study involving 310 men and 492 women from
the Framingham Osteoporosis Study with 2 measures of femoral neck BMD taken
from 1987 through 1999.
MAIN OUTCOMES AND MEASURES
Risk of hip or major osteoporotic fracture through 2009 or 12 years
following the second BMD measure.
Mean age was 74.8 years. The mean (SD) BMD change was
−0.6% per year (1.8%). Throughout a median follow-up
of 9.6 years, 76 participants experienced an incident hip fracture and 113
participants experienced a major osteoporotic fracture. Annual percent BMD
change per SD decrease was associated with risk of hip fracture (hazard
ratio [HR], 1.43 [95% CI, 1.16 to
1.78]) and major osteoporotic fracture (HR, 1.21
[95% CI, 1.01 to 1.45]) after adjusting for baseline
BMD. At 10 years’ follow-up, 1 SD decrease in annual percent BMD
change compared with the mean BMD change was associated with 3.9 excess hip
fractures per 100 persons. In receiver operating characteristic (ROC) curve
analyses, the addition of BMD change to a model with baseline BMD did not
meaningfully improve performance. The area under the curve (AUC) was 0.71
(95% CI, 0.65 to 0.78) for the baseline BMD model compared with 0.68
(95% CI, 0.62 to 0.75) for the BMD percent change model. Moreover,
the addition of BMD change to a model with baseline BMD did not meaningfully
improve performance (AUC, 0.72 [95% CI, 0.66 to
0.79]). Using the net reclassification index, a second BMD measure
increased the proportion of participants reclassified as high risk of hip
fracture by 3.9% (95% CI, −2.2% to
9.9%), whereas it decreased the proportion classified as low risk by
−2.2% (95% CI, −4.5% to
CONCLUSIONS AND RELEVANCE
In untreated men and women of mean age 75 years, a second BMD measure
after 4 years did not meaningfully improve the prediction of hip or major
osteoporotic fracture. Repeating a BMD measure within 4 years to improve
fracture risk stratification may not be necessary in adults this age
untreated for osteoporosis.
The primary objective of this multicenter registry was to study the prognostic value of PET MPI and the improved classification of risk in a large cohort of patients with suspected or known coronary artery disease (CAD).
Limited prognostic data are available for myocardial perfusion imaging (MPI) with positron emission tomography (PET).
7,061 patients from 4 centers underwent a clinically indicated rest/stress rubidium-82 PET MPI with a median follow-up of 2.2 years. The primary outcome of this study was cardiac-death (169 patients) and the secondary outcome was all-cause death (570 patients). Net reclassification improvement (NRI) and integrated discrimination (IDI) analyses were performed.
Risk-adjusted hazard of cardiac-death increased with each 10% abnormal myocardium with mildly, moderately or severely abnormal stress PET [hazard ratio 2.3 (95% CI 1.4–3.8, P=0.001), 4.2 (95% CI 2.3–7.5, P<0.001), and 4.9 (95% CI 2.5–9.6, P <0.0001), respectively, normal MPI: referent]. Addition of %myocardium ischemic and scarred to clinical information (age, female sex, body mass index, history of hypertension, diabetes, dyslipidemia, smoking, angina, betablocker use, prior revascularization and rest heart rate) improved the model performance [C-statistic 0.805 (95% CI, 0.772–0.838) to 0.839 (95% CI, 0.809–0.869)] and risk reclassification for cardiac-death [NRI 0.116 (95% CI 0.021–0.210)] with smaller improvements in risk assessment for all-cause death.
In patients with known or suspected CAD, the extent and severity of ischemia and scar on PET MPI provide powerful and incremental risk estimates of cardiac-death and all-cause death compared to traditional coronary risk factors.
positron emission tomography; registry; prognosis; myocardial perfusion imaging; risk reclassification