In this cohort of older community dwelling women, a simple model based on age and BMD alone predicted 10-year risk of hip, major osteoporotic, and any clinical fracture as well as more complex FRAX® models with BMD. Similarly, a parsimonious model based on age and fracture history alone predicted 10-year risk of these three fracture outcomes as well as more complex FRAX® models without BMD. These findings suggest that use of the FRAX® risk assessment tool does not enhance fracture prediction in older women beyond that provided by simple models based on age and BMD or age and fracture history alone.
Our results are in general agreement with published findings regarding the predictive validity of the FRAX® risk assessment tool. In the FRAX® development and validation study4
, findings concerning hip fracture prediction indicated that a model combining BMD with FRAX® clinical risk factors vs. a model with BMD alone resulted in a small increment (5 to 19% increase depending on age group) in the gradient of risk per one standard deviation change in risk score. However, the performance of the model with BMD alone (assessed with ROC analysis) was nearly identical to that for the model with BMD plus FRAX® clinical risk factors suggesting that hip fracture prediction did not substantially improve by adding FRAX® clinical risk factors to the model with BMD alone. There was also a small increment (11 to 21% increase depending on age group) in the gradient of risk for prediction of other osteoporotic fractures (defined variably depending on the cohort, excluded hip fractures) when FRAX® clinical risk factors were used in combination with BMD; however, a comparison of the performance of the model containing BMD plus clinical risk factors with that of the model with BMD alone was not reported. Findings from this study and the FRAX® development and validation study both suggest that models for predicting hip fracture perform better than those for predicting other fracture outcomes. Neither this study nor the FRAX® development and validation study examined the ability of models to predict radiographic vertebral fracture, the most common manifestation of osteoporosis. However, a study using data collected in the Fracture Intervention Trial26
reported that the FRAX® model with BMD and a simple model based on age and BMD alone were similarly accurate in terms of discrimination of new radiographic vertebral fracture.
While gradients of risk and areas under ROC curves are widely utilized methods for model comparison, the clinical usefulness of a risk prediction model may also be judged by the extent to which the risk calculated from the model reflects the fraction of individuals in the population who actually sustained the outcome of interest. To address this issue, we calculated the proportion of women in each quartile (and decile) of predicted risk who actually experienced a fracture outcome using FRAX® and simple models and compared these proportions between models. These results also suggest that use of FRAX® models does not enhance fracture prediction beyond that provided by more parsimonious models as there was no evidence that use of the FRAX® models improved the classification of high (and low) risk categories such that a higher (and lower) proportion of women who actually experienced a fracture outcome were identified. Similar findings have been reported in the study comparing FRAX® and simple models for the prediction of new radiographic vertebral fracture.26
The FRAX® risk assessment tool represents a major advance in the field of osteoporosis for several reasons. The tool is based on data collected from cohorts in the USA, Europe, Australia and Asia and is applicable to both the developed and developing world. Modeling techniques incorporated into the FRAX® tool take into account country-specific fracture and death rates. Its aim to move forward risk assessment from a strategy based on BMD alone to an approach based on the absolute risk of fracture is appealing as absolute risk classification systems overcome several of the drawbacks posed by relative risk classification systems and may be more intuitive to both clinicians and patients.27
However, despite these merits, findings from this study in older Caucasian women suggest that use of the FRAX® tool does not lead to substantial improvements in fracture risk prediction. Cost-effective analyses16
supporting use of the FRAX® tool to select men and women aged 50 years and older with osteopenia for pharmacologic therapy rely on a critical, but controversial assumption28-31
that drug treatment is effective in reducing the risk of all clinical fractures regardless of BMD status. Moreover, application of the new NOF guidelines may result in recommending pharmacologic treatment to a very large proportion of women aged 65 years and older.20
Thus, randomized trials evaluating the effectiveness of a treatment approaches based on absolute risk of fracture including the approach recommended by NOF (incorporating selected risk cutpoints based on FRAX® tool probabilities) should be conducted prior to widespread implementation of these approaches into clinical decision making regarding whether or not to initiate drug therapy.
This study has a number of strengths including the comprehensive set of measurements and duration and completeness of follow-up. However, this study has several limitations. Our findings are based on a single cohort of older Caucasian women. Although we cross-validated our findings regarding the performance of models, results concerning the predictive validity of FRAX® and simple models require confirmation in other cohorts. In particular, these findings may not be generalizable to younger women or men. Other than rheumatoid arthritis, data on six specific conditions associated with secondary osteoporosis that comprise an additional component in the FRAX® risk assessment tool were not collected in SOF. However, these medical conditions are uncommon in healthy older women and their association with increased fracture risk is in large part due to lower BMD among those with disease. For this reason, checking the “secondary osteoporosis” box in FRAX® does not alter the risk score once BMD is entered into the algorithm.32
For a given patient, FRAX® and simple models will not necessarily agree on the classification of risk status despite nearly identical ROC curves. Other methods are available to assess and interpret the utility of a new model such as measures of reclassification33;34
that determine how often an alternative model successfully reclassifies individuals from one risk class to another and thus alters individual treatment decisions. However, the use of these measures require the existence of predefined treatment thresholds at which treatments would be altered, as well as the availability of effective treatments at different risk levels. In the case of osteoporosis, these requirements are not met at present, especially the availability of treatments at different risk levels. While model weights in the parsimonious models were derived from the SOF population and the potential for over-fitting exists, findings from this study are consistent with those reported in the FRAX® development and validation study14
and other cohorts.26
Since the equations and algorithms that generate the fracture risk probabilities have not been published for the FRAX® tool, it was not feasible to directly quantify the change in the AUC statistic which accompanies adding each component of the FRAX® model. Finally, both the FRAX® models and simple models are limited in their ability to predict fracture, especially nonhip fractures. Thus, risk prediction is challenging, even when robust risk factors like BMD and age are available.
We conclude that a simple model based on age and BMD alone predicted 10-year risk of hip, major osteoporotic, and any clinical fracture as well as more complex FRAX® models with BMD. Similarly, a parsimonious model based on age and fracture history alone predicted 10-year risk of these fractures as well as more complex FRAX® models without BMD. These results suggest that use of the FRAX® risk assessment tool does not improve fracture prediction in older women beyond that provided by simple models based on age and BMD or age and prior fracture alone.