|Home | About | Journals | Submit | Contact Us | Français|
The primary prevention of cardiovascular disease involves using the Framingham risk score to identify high risk patients and then prescribe preventive treatments.
To examine the performance of the Framingham risk score in different socioeconomic groups in a population with high rates of cardiovascular disease.
A prospective study.
West of Scotland.
The observed 10-year cardiovascular disease and coronary heart disease mortality rates in 5626 men and 6678 women free from cardiovascular disease from the Renfrew/Paisley Study were compared with predicted rates, stratified by socioeconomic class and by area deprivation score.
The ratio of predicted to observed cardiovascular mortality rate in the 12 304 men and women with complete risk factor information was 0.56 (95% confidence interval [CI] = 0.52 to 0.60), a relative underestimation of 44%. Cardiovascular disease mortality was underestimated by 48% in manual participants (predicted over observed = 0.52, 95% CI = 0.48 to 0.56) compared to 31% in the non-manual participants (predicted over observed = 0.69, 95% CI = 0.60 to 0.81, P = 0.0005). Underestimation was also worse in participants from deprived areas (P = 0.0017). Only 4.8% of individuals had a 10-year cardiovascular risk of >40% (equivalent to >30% 10-year coronary risk), and 81% of deaths occurred in the rest. If the Framingham score had been recalibrated for manual and non-manual members of this population, an additional 3611 individuals mainly from manual social classes would have reached the treatment threshold.
Currently recommended risk scoring methods underestimate risk in socioeconomically deprived individuals. The likely consequence is that preventive treatments are less available to the most needy.
The National Service Framework for coronary heart disease in England and Wales states that individuals whose estimated coronary heart disease risk is 30% or more over 10 years should be identified and offered appropriate advice and treatment. An assessment of absolute cardiovascular risk is also integral to recent hypertension guidance.1–3 Scottish and international guidance also recommend calculating an individual's risk based upon regression equations derived from the Framingham Heart Study and Framingham Offspring Study.4–9
Recent studies have shown that Framingham risk scores overestimate coronary risk in British and other European populations, with growing international evidence that the accuracy of the Framingham risk equations depends upon the background risk of the population to which they are applied.10–14 Cardiovascular disease risk is greater in low-income populations for reasons that cannot be explained by classical risk factors, so the Framingham risk score may perform differently in such populations.15,16 If, because of this, their predicted risk fails to reach the treatment threshold, patients in poor socioeconomic circumstances may be excluded from receiving preventive treatment. In this paper, we assess the ability of the Framingham risk score to predict 10-year cardiovascular and coronary heart disease death in individuals from different social classes and who are resident in different categories of deprivation, ranging from affluent to deprived.
Between 1972 and 1976, 15 406 men and women aged 45–64 years from the general population of Renfrew and Paisley in the west of Scotland were screened for cardiovascular risk factors. The response rate was 78% and full details of the study have been published elsewhere.17,18 Social class was determined by regular occupation. In the case of retired men, the last full-time occupation was recorded. For housewives, their husbands' occupations were used. Participants were classified as non-manual if they were in social classes I, II, IIIN and manual if in social classes IIIM, IV, or V. The postcode of the home address at the time of screening enabled the Carstairs and Morris deprivation category ranging from 1 to 7, to be ascertained.19 Participants were categorised into three groups, defined as Affluent (1–3), Intermediate (4–5) or Deprived (6–7). Subjects were excluded if they had a history of cardiovascular disease or if data on risk factors or socioeconomic deprivation were missing (Table 1). Data on high-density lipoprotein cholesterol were not collected, so the mean values of 1.3 mmol/l for men and 1.5 mmol/l for women from the Scottish Health Survey were used.20 Participants were flagged at the General Register Office in Edinburgh and cardiovascular disease and coronary heart disease deaths were identified over 10 years of follow up.
The risk assessment methods recommended for British and European use are adapted from published equations derived from 5573 men and women, free of cardiovascular disease, from the Framingham Heart Study and the Framingham Offspring Study.9,21 Prediction equations relating to six outcomes have been derived, and these equations can be used to estimate risk over a range of 4–12 years for persons aged 30–74 years. Two of these outcomes are considered in this paper: death from cardiovascular disease and death from coronary heart disease.9
Guidelines for the primary prevention of cardiovascular disease recommend using the Framingham risk score to identify high-risk patients and prescribing preventive treatments. The Framingham risk score is known to overestimate cardiovascular disease risk in some British and other European populations but its performance in different socioeconomic groups is unknown. In this study the Framingham risk score underestimated the risk of dying from cardiovascular disease in men and women from a population with high levels of socioeconomic deprivation and high rates of cardiovascular mortality. This underestimation was worse in people from areas of high deprivation and from manual social classes, relative to more affluent individuals, leading to fewer people in deprived areas reaching current treatment thresholds. Future risk assessment methods could include measures of socioeconomic deprivation to improve the targeting of preventive treatment to those who need it the most.
Predicted versus observed 10-year risk: The predicted probabilities of cardiovascular disease and coronary heart disease mortality within 10 years were calculated for each participant using the relevant Framingham equations.9 The average predicted mortality rates within each category of deprivation (affluent, intermediate and deprived) and social class (non-manual and manual) were compared with the observed 10-year rates calculated by the Kaplan–Meier method.22 Participants were further categorised into groups defined by quintiles of predicted risk calculated using the Framingham cardiovascular disease event equation rather than the mortality equations, as this is the outcome recommended for risk assessment in the most recent guidance.9
To assess the ability of the Framingham equation for cardiovascular disease risk to rank individuals relative to their observed risk of cardiovascular and coronary heart disease death within 10 years, the area under the receiver operating characteristic (AUROC) curves for both endpoints within each social class group was calculated. In recent guidance, the decision to prescribe preventive treatment depends upon thresholds of fatal and non-fatal cardiovascular risk.2,3 We therefore calculated the sensitivities, specificities, positive and negative predictive values, and the positive and negative likelihood ratios for each social class group at the risk score thresholds of 40, 30, 20 and 10% cardiovascular disease event risk within 10 years. The likelihood ratio is a clinically useful measure that combines sensitivity and specificity and expresses the power of a test result to change the odds of disease.
We used bootstrap techniques for calculating 95% confidence intervals for the predicted versus observed risk ratio, the likelihood ratios and the tests for trends and differences between subgroups.23
Of the 15 406 men and women recruited to the Renfrew/Paisley study, 2648 (17.2%) with a history of cardiovascular disease were excluded. A further 454 with incomplete risk factor information were excluded, leaving a total of 5626 men and 6678 women for analysis. Table 2 compares the baseline characteristics of these participants with the 2590 men and 2983 women from the Framingham study used in the derivation of the risk equations.
The predicted 10-year cardiovascular disease mortality rates for men and women were 4.7% and 2.2%, respectively, compared with the observed rates of 8.2% and 4.0% leading to predicted/observed ratios of 0.57 (95% confidence interval [CI] = 0.53 to 0.63) for men and 0.55 (95% CI = 0.49 to 0.62) for women. The equivalent predicted/observed ratios for coronary heart disease mortality were 0.66 (95% CI = 0.60 to 0.74) for men and 0.53 (95% CI = 0.46 to 0.63) for women. The underprediction for men and women combined was 44% (predicted over observed 0.56, 95% CI = 0.52 to 0.60) for cardiovascular disease mortality and 38% (predicted over observed 0.62, 95% CI = 0.57 to 0.68) for coronary heart disease mortality.
The effects of area deprivation and social class upon the accuracy of the Framingham score were independent of the sex of the participants, so we combined the results for men and women. The Framingham score under predicted cardiovascular disease risk in non-manual participants by 31% (predicted over observed 0.69, 95% CI = 0.60 to 0.81) compared to 48% in the manual participants (predicted over observed 0.52, 95% CI = 0.48 to 0.56, P-value for the difference =0.0005) (Table 3). There was a trend of worsening underprediction of cardiovascular disease, from participants in affluent areas to those from deprived areas (P = 0.0017). These patterns were similar when the analysis was restricted to coronary heart disease mortality, with a 22% underprediction in non-manual participants, compared to 44% for manual participants (P <0.0001), although the trend for underprediction in in deprived areas relative to affluent areas did not reach statistical significance (P = 0.067).
The relative underprediction of cardiovascular disease risk was similar between quintiles of Framingham risk although less pronounced in the lowest and highest risk quintiles. (Figure 1).
Applying the Framingham cardiovascular disease event equation to the Renfrew and Paisley baseline data, 585 (4.8%) individuals were identified as being at >40% 10-year cardiovascular disease risk and 18.8% of the cardiovascular deaths occurred in this group. Forty per cent 10-year cardiovascular disease risk is roughly equivalent to the 30% 10-year coronary heart disease threshold recommended by earlier guidance.1,24 Nearly 43% of participants were above the 20% threshold (equivalent to 15% coronary heart disease), and the sensitivity for identifying cardiovascular disease death increased to 74.3%, at the expense of the false positive rate increasing from 3.9% to 40.6%.
To estimate the numbers of participants who failed to reach these treatment thresholds because of the underestimation of their true risk, we inflated each participant's predicted risk, multiplying it by the average underprediction of the social class group to which they belonged. Inflating participants’ predicted risks of cardiovascular disease death in this way resulted in 4196 (34.1%) individuals reaching the >40% threshold. The percentage reaching the >20% threshold increased from 42.6% to 77.0% after adjustment. Table 4 shows the effect of underprediction in each social class group, with 7.9 times more manual individuals reaching the >40% threshold after adjustment and a 4.9-fold increase following adjustment in the non-manual participants.
Using the standard unadjusted equation, the area under the receiver operating characteristic curve was 74.4 (95% CI = 71.0 to 77.7) for the non-manual and 72.0 (95% CI = 69.9 to 74.1) manual groups. Table 5 shows the expected worsening of the positive likelihood ratio and improving negative likelihood ratio at the lower risk thresholds. An individual who is >40% is about 4.8 times more likely to die from cardiovascular disease than someone with a risk of <40%. A negative test at this threshold is, however, almost useless at reassuring individuals. At each threshold, the positive likelihood ratio is less in the manual compared to the non-manual social class although the P-value is high at the >40% threshold representing the small numbers of very high-risk participants. There is no difference between social classes in the discriminatory power of a negative test at different thresholds as indicated by the negative likelihood ratio.
High-density lipoprotein cholesterol was not measured in the Renfrew and Paisley study, and as the total cholesterol to high-density lipoprotein-cholesterol ratio is included in the Framingham equations, we used default measures likely to represent the average high-density lipoprotein levels of the participants.20 Although there is no relationship between high-density lipoprotein and social class in men,20 we performed sensitivity analyses using high-density lipoprotein values 15% above and below the default values. The pattern and scale of worsening underestimation with increasing socioeconomically disadvantage remained unchanged.
The Renfrew and Paisley Study had a lower prevalence of diabetics than the Framingham Study, that may be due to the use of different definitions of diabetes in the two studies. We performed a sensitivity analysis involving changing the diabetic status from ‘not diabetic’ to ‘diabetic’ in a randomly selected sample of Renfrew and Paisley participants to achieve the same sex-specific prevalence of diabetes as in the Framingham study. This had no significant effect on the results, probably because of limited influence of the small numbers of diabetics on the results
This paper shows that the Framingham score underestimates the risk of cardiovascular disease death in men and women from a population with high levels of socioeconomic deprivation and high rates of cardiovascular mortality. The underestimation is significantly greater in people from manual social classes and who live in deprived areas, than in more affluent people. Consequently, targeting preventive treatment using thresholds of Framingham risk as a screening test was poor, with a large proportion of people, especially in manual social classes, failing to reach treatment thresholds. Disproportionate underestimation of risk may contribute to health inequalities, as individuals might not be identified as high risk. A likely consequence is that cardiovascular disease risk-reducing treatments are not being offered to those who need them most.
Our results suggest that individual and area-based measures of socioeconomic deprivation might make an additional contribution to cardiovascular risk prediction in addition to the traditional risk factors included in the Framingham equation. This may be because measures of social deprivation represent other risk factors such as early life and childhood growth, diet, exercise and other lifestyle factors influenced by socioeconomic position and area of residence or differences in preventive or hospital treatment experienced by people from different backgounds.25 The use of dichotomous smoking and diabetes variables may also fail to capture subtle differences between socioeconomic groups.
As far as we are aware, this is the first study to examine the accuracy of Framingham risk prediction in people from different social classes and from areas of different deprivation index. The Renfrew and Paisley Study population is typical of some of the most deprived areas of Scotland and within this area there is still considerable heterogeneity in the distribution of area-based deprivation and individual social class.25 The observation of increasing underestimation of cardiovascular risk with worsening socioeconomic position is likely to be robust, as it is present using both an individual measure of social class and an area-based indicator of deprivation.
We have found the same findings using two mortality endpoints. Using cardiovascular disease mortality as the primary outcome for the study is relevant, as recent risk-scoring tools have adopted this outcome in preference to coronary heart disease.8,26,27 However, recent hypertension guidelines recommend the more inclusive fatal and non-fatal cardiovascular disease as the chosen endpoint.2,3 We were unable to compare predicted disease rates with observed rates for Framingham endpoints that include non-fatal events, as the Renfrew/Paisley and the Framingham studies ascertained non-fatal events differently. However, where a difference in predicted Framingham risk and observed outcomes has been found, it is similar for both fatal and non-fatal events.11
Although we are not aware of other studies that have examined the accuracy of Framingham predictions in people with different levels of social deprivation, several studies have shown that the ability of the Framingham equations to predict absolute risk is dependent upon the background risk of the population to which they are applied.10–14,28 One of these set in Whickham, north-east England, found that the Framingham score underestimated risk in individuals with an annual coronary risk of <1.5%, while another British study concluded that the Framingham score predicted reasonably well in the placebo arm of a trial population.29,30 A third British study, in contrast to the our findings, found that the Framingham score overestimated non-fatal and fatal coronary heart disease events by about 50% in a sample of men from the British Regional Heart Study.11 The difference in these results might be partially explained by much lower 10-year coronary heart disease mortality rate of 4.1%, in the British Regional Heart Study compared with 6.3% for the men of the Renfrew/Paisley study. Studies in populations with lower disease rates, have consistently found that Framingham overestimates coronary and cardiovascular disease risk.10–13
The underestimation of cardiovascular risk by the Framingham score in this high risk population, with worse performance in lower income populations relative to the more affluent, has important implications for patients, their doctors and public policy. The continued use of the Framingham risk score in its current form has the potential to contribute to existing health inequalities. Fewer individuals from deprived backgrounds may reach treatment thresholds because the risk score fails to take into account the additional risk associated with where people live or their socioeconomic position. The number of cardiovascular deaths prevented could be increased if treatment is better targeted by a more accurate and inclusive risk assessment method.
There is increasing evidence that the Framingham risk equations are unable to provide accurate estimations of absolute risk in individuals from different populations. Risk estimates derived from cohorts studies such as Framingham and the recent SCORE project, do not have the flexibility to incorporate regional, socioeconomic and temporal differences in disease rates.26 Alternative approaches to risk assessment and treatment are required that might be more adaptable or applied differently to populations with varying risk factor and disease patterns.
Peter M Brindle was funded by the Wellcome Trust