In this nationally representative study, the original and point-based Framingham models produced clinically meaningful differences in estimated CHD risk for many individuals and stratified substantial numbers of patients into different risk groups established by ATP III guidelines. Overall, the point-based system classified 15% of eligible Americans (5.7 million people) into different risk groups than the original Framingham model. Misclassification predominantly shifted patients into higher risk groups, with 10% of adults (3.9 million) misclassified into higher risk groups and 5% (1.8 million) into lower risk groups, and had the potential to impact drug treatment recommendations in 25–46% of affected subjects not currently taking lipid-lowering therapy. Because our analyses excluded study subjects with incomplete data, patients on lipid-lowering therapy, and patients who in clinical practice may receive Framingham risk prediction outside formal guideline criteria, our results underestimate the number of Americans potentially affected by differences in the point-based and original Framingham models.6
These discrepancies comprise one of the ongoing challenges in a history of impressive advances in cardiovascular risk assessment. Beginning with a sum-of-risk-factors approach in the first report of the National Cholesterol Education Program, successive advances in modeling have improved clinicians’ ability to predict—and thereby better prevent—cardiovascular events.15–18
As predictive models became more complex and impractical to calculate by hand, point-based versions became necessary to facilitate their regular use. This need may persist in settings where computer-based risk calculators are not readily available at the point of care, as it may be preferable to have an imperfect system of risk prediction than none at all. However, as the availability of desktop- and handheld-based computers has become routine in clinical practice, there is limited need for predictive models that can be calculated using pen and paper.
Nonetheless, the point-based system remains in widespread use, including in risk calculators on websites and personal digital assistants, and such tools are often not transparent in noting which model they use. Thus, the misclassifications of risk that we observed are likely common in clinical practice, and may have substantial clinical and policy implications. Of particular note, over two-thirds of misclassifications moved patients into higher risk groups. Because guidelines recommend more aggressive treatment strategies for patients in higher risk groups, this misclassification may drive increases in the use of lipid-lowering medications. This may have some benefits by reducing cardiovascular event rates, although at the risk of increasing adverse drug reactions, patients’ medication burden, and clinician time and resources.19
In addition, there is debate over the utility of expanding drug therapy beyond NCEP guidelines.3,20–30
Also concerning is potential undertreatment for the 1.8 million people whom the point-based system misclassifies into lower risk groups, particularly the nearly 50% of people (1.2 million) at high coronary risk whom the point-based system triages into lower risk categories. Failure to define and pursue aggressive LDL goals in such patients may compound the widespread undertreatment of persons at high cardiovascular risk.10
Unfortunately, there does not appear to be a simple “fix” to correct the misclassification that occurred under the point-based system. Patterns of misclassification were complex, varying by underlying CHD risk, sex, and age. In addition, the population-level implications of misclassification also vary among age and sex groups. For example, a substantial majority (80%) of the 1.4 million misclassified women were misclassified into higher risk groups, largely reflecting the fact that most women had calculated risk of under 10% by the original model, so their only available direction for risk group misclassification was into a higher risk group. In contrast, while misclassification also predominantly placed men into higher risk groups, a substantial minority (36%) were shifted into lower risk groups, leaving them susceptible to undertreatment. This was particularly notable for the 2.5 million men at high risk of future coronary events, almost half of whom were misclassified by the point-based model into a lower risk group.
ATP III guidelines acknowledge that the original model gives more precise estimates of risk than the point-based one, but note that use of the point-based system “provide[s] a result that is accurate for clinical purposes.”31
While the differences in predicted risk between the models are small for the majority of patients, there are substantial numbers of patients for whom the two models produce clinically meaningful differences in predicted risk. National guidelines would benefit from acknowledging the calculated discrepancies between the two models and educating and guiding clinicians about preferred methods of risk stratification. More importantly, current guidelines should strongly consider endorsing the original model as the preferred method of risk calculation and as the sole appropriate option for computer or PDA-based risk calculators. In addition, patients and clinicians who made treatment decisions based on the point-based system should consider recalculating risk based on the original Framingham model and where appropriate adjust treatment plans accordingly.
Our results should be interpreted in the context of known limitations of the original Framingham model and previous evaluations of the NCEP risk stratification algorithm.6,7,32
The original Framingham model has only moderate ability to distinguish between persons who will or will not have future coronary events (with ROC curves from validation studies mostly in the range of 0.65 to 0.75).33–36
This model was also developed in a mostly white, middle-class population, and validation studies have revealed that it overestimates CHD risk in a number of other populations.33–36
Other research suggests that Framingham-based risk assessment should be expanded to patients with 0 or 1 risk factors.6,7
In addition, national guidelines from other countries use versions of the Framingham model in different, often more conservative ways to guide lipid management.37
In the US, it is well-documented that many patients—particularly those at high coronary risk—have LDL levels above current guideline recommendations.10
Thus, our findings should be interpreted as one piece of a larger challenge of appropriately identifying individuals’ coronary risk profiles and increasing adherence to treatment strategies optimally tailored to those individuals’ risk.
The next generation of cholesterol guidelines (ATP IV) is expected to be released in the near future, and it is likely that these new guidelines will predict risk using a new model of global cardiovascular risk prediction that incorporates a broader range of cardiovascular outcomes.35
Nonetheless, score-sheet versions of this model have already been developed and if applied to guidelines may result in problems similar to those that we observed.35
Thus, when simplifying future models of cardiovascular or other forms of risk, it will be essential to account for the practical effects of simplification on algorithm-based management decisions and to disseminate these analyses in peer-reviewed publications to maximize transparency.38
There are several limitations to our study. Our estimates of how many subjects would be recommended for changes in lipid-lowering therapy based on misclassification are approximate due to limited sample sizes, absence of data on potential lifestyle interventions, and potential inaccuracies in self-reported use of lipid-lowering medications. In addition, we did not evaluate users of lipid-lowering drugs, so we do not know what the impact of using different Framingham models would have been prior to their initiating drug treatment. Finally, our study did not have access to actual cardiovascular outcomes, and so we are unable to determine the accuracy of these models for predicting cardiovascular events. Nonetheless, the original Framingham model is the de facto gold standard for ATP III-based risk prediction, and mathematically is it very unlikely that a point-based system derived from the original model would be more accurate than the original model itself.
In summary, the point-based Framingham risk prediction tool misclassifies millions of Americans into different ATP III risk groups compared with the original Framingham model, with 25–46% of affected subjects experiencing potential impacts on drug treatment recommendations. Guidelines and their associated risk prediction tools should account for the clinically meaningful differences that can arise between original and point-based models and the impact that these differences can have on treatment decisions. This will support the goal of a clinically consistent, transparent, and standardized approach to cardiovascular risk assessment.