One of the most important findings to come out of the landmark Framingham Heart Study was the development of tools to predict patients’ risk of developing coronary artery disease (CHD).1 By analyzing data across multiple CHD risk factor categories derived from this prospective cohort, it became possible to predict patients’ 10-year risk for CHD; the prediction model was based on a set of parameter estimates for each risk factor and combinations of risk factors.2 Wilson et al. recognized if physicians were to use these tools for clinical decision-making that they needed to be quick and easy to use. In 1998, they published the sex-specific Framingham Risk Scoring (FRS) system.3 FRS involved converting the parameter estimates from the original prediction model into whole numbers (points) that could then be summed and converted into a 10-year risk. Scoring algorithms using these points are available in print, online and through hand-held electronic devices. Use of FRS point scoring was promulgated in the Third Adult Treatment Panel (ATP III) recommendations for treatment of high cholesterol by the National Cholesterol Education Program.4 FRS arguably represents the most widely used validated-tool for personalized clinical decision making.
In a provocative study, Gordon et al. compared the risk classification of FRS point scoring with the original Framingham prediction model using risk factor prevalence information derived from the National Health and Nutrition Survey (NHANES) 2001–2006.5 They observed significant discrepancies between the two methods. Relative to the original model, point counting reclassifies a large percentage of patients into risk categories that could potentially alter treatment eligibility based on ATP III recommendations; notably, use of the counting system instead of the original parameter estimates resulted in 5% of patients being “under-treated” and 10% being “over-treated” (assuming the original parameter estimates represent the gold standard). While the authors do not compare the actual performance of these tools, they argue that it is implausible that a simplified version in which parameter estimates are rounded to the nearest whole number would outperform the original, more sophisticated version. They further suggest that widespread use of electronic devices including those loaded with FRS software make it feasible to use the original Framingham model to estimate risk, essentially rendering point scoring a relic of the pre-electronic age.
Are their findings valid? Consistent with ATP III guidelines, Gordon et al. excluded those with low risk and known CHD (or CHD equivalent) disease from the analysis, thus sharpening the contrast between the two models by focusing on those in the intermediate risk range. Patients taking lipid-lowering therapy were excluded because FRS was not designed to assess risk among persons receiving drug treatment. However, this exclusion creates a non-representative sample. Potentially, persons who were treated based on use of FRS point scoring were excluded from further analysis. The impact of this bias is hard to quantify, but given that only 11% of the study sample were affected, it seems unlikely that this exclusion would significantly alter their findings, particularly among those in the intermediate risk categories where drug treatment is less frequent. Gordon et al. confined their sample to those with two or more major risk factors. Had they included all untreated adults with complete data and compared the original FRS model to point scoring using for the ATP III two-step procedure, even greater discrepancies may have been noted as suggested by an analysis of earlier NHANES.6 Thus, on balance it appears that the authors’ conclusion regarding significant discrepancies in CHD risk or “misclassification” is justified.
Are their recommendations sound? Do these observed discrepancies affect patient outcomes? Is it time to incorporate the original Framingham prediction model into electronic versions of FRS? Studies have suggested that the proper use of assessment tools improves outcomes. In the absence of formal risk assessment, physicians often misjudge CHD risk.7 A meta-analysis showed that CHD risk assessment improves patient outcomes and reduces harm.8 Nonetheless, Wilson et al.’s hope that most physicians would actually use FRS point scoring has not been realized. In fact, most physicians do not use any type of FRS.9,10 Furthermore, even when CHD risk scores are appended to patients’ charts, most physicians do not alter their clinical decision-making.11 In the context of current medical practice, discrepancies between these models may have relatively little impact on patient outcomes because physicians typically do not use FRS point counting or FRS-based on more precise parameter estimates.
But, medical practice is at the cusp of an electronic transformation aided by rapidly evolving technology, recent federal standards, and financial incentives for “meaningful use” of electronic health records (EHRs).12 The recent appointment of health care improvement maven Donald Berwick to head the Centers for Medicare and Medicaid Services may also herald the acceleration of the use of technology to improve quality. Adoption of EHRs by community physicians is rapidly approaching the tipping point. By 2009, 44% of office physicians in the US reported use of some type of EHR.13 EHRs make it technically feasible to automatically generate CHD risk assessments and identify patients in need of interventions.14,15 Increasingly, EHR data will also be used to monitor (and potentially improve) physician performance. These same data will also be fodder for malpractice claims. Physicians who choose to deviate from embedded clinical decision support systems that include risk assessment and corresponding specific clinical recommendations will be expected to document their reasons for doing so.
In this rapidly evolving environment for medical practice, simple point scoring systems may have seen their day. As Gordon et al. suggest, EHRs and portable electronic devices enable risk tools that require complex computations. EHRs could also facilitate tailoring risk assessment-based social and other novel risk factors. For example, consideration of SES improves risk calibration, particularly for low socioeconomic status persons whose risk is underestimated by FRS.16 Recent health reform legislation strengthening requirements for collection of patient sociodemographic information coupled with increasing use of area-based socioeconomic measures will facilitate consideration of a range of CHD risk factors when personalizing risk.
The above scenario is not farfetched. The UK, perhaps a decade or so ahead of the US in EHR adoption in primary care, provides a glimpse of future practice in the US.17 SES is incorporated into the most widely used risk assessment tool, QRISK2.18 Moreover, widespread use of EHRs in the UK has facilitated establishment of prospective patient cohorts that have yielded improved cardiovascular risk tools such as QRISK2. These cohorts, often comprising more than a million patients, and unimaginable in the pre-electronic age, have made it possible to model CHD risk for relatively small ethnic minority groups.19 These huge samples derived from hundreds of networked practices, also facilitate individualized estimates of the benefits and harms associated with statin treatment.20 Establishment of similar comparative effectiveness research cohorts in the US will enable far more precisely personalized risk assessment and treatment guidelines than has been heretofore possible. Thus, Gordon et al.’s suggestion to begin using the best available FRS prediction models with currently available electronic devices seems timely.