Empirical Derivation to Improve the Definition of the Metabolic Syndrome in the Evaluation of Cardiovascular Disease Risk

doi:10.2337/dc10-1715

Diabetes Care. 2011 March; 34(3): 746–748.

Published online 2011 February 17. doi: 10.2337/dc10-1715

PMCID: PMC3041220

Empirical Derivation to Improve the Definition of the Metabolic Syndrome in the Evaluation of Cardiovascular Disease Risk

Rachel P. Wildman, PHD,¹ Aileen P. McGinn, PHD,¹ Mimi Kim, PHD,¹ Paul Muntner, PHD,² Dan Wang, MS,¹ Hillel W. Cohen, DRPH,¹ Alexandra D. Ogorodnikova, MS,¹ Kristi Reynolds, PHD,³ and Vivian Fonseca, MD⁴

¹Department of Epidemiology and Population Health, Albert Einstein College of Medicine, Bronx, New York

²Department of Epidemiology, School of Public Health, University of Alabama at Birmingham, Birmingham, Alabama

³Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, California

⁴Section of Endocrinology, Department of Medicine, Tulane University School of Medicine, New Orleans, Louisiana

Corresponding author: Rachel P. Wildman, Email: rachel.wildman/at/einstein.yu.edu.

Received September 3, 2010; Accepted December 22, 2010.

Readers may use this article as long as the work is properly cited, the use is educational and not for profit, and the work is not altered. See http://creativecommons.org/licenses/by-nc-nd/3.0/ for details.

Abstract

OBJECTIVE

To examine whether a quantitatively derived metabolic syndrome definition predicts incident cardiovascular disease (CVD) events better than do existing definitions.

RESEARCH DESIGN AND METHODS

Data were pooled from the Atherosclerosis Risk in Communities, Cardiovascular Health, and Framingham Offspring studies (n = 20,581). Incident coronary heart disease and stroke events were ascertained over 9 years.

RESULTS

The sensitivity for incident CVD events was higher and the specificity lower for the empirically derived versus the Adult Treatment Panel (ATP) III, International Diabetes Federation (IDF), or Harmonized metabolic syndrome definitions (sensitivity/specificity 0.65/0.53 vs. 0.53/0.63, 0.51/0.66, and 0.64/0.56, respectively), resulting in no overall improvement in discrimination. Multivariable-adjusted hazard ratios for incident CVD events were similar across definitions and were 1.7 (95% CI 1.6–1.9) for ATP III, 1.8 (1.6–2.0) for IDF, 1.9 (1.7–2.0) for Harmonized, and 1.7 (1.6–1.9) for the empirically derived definition.

CONCLUSIONS

Empirical derivation of the metabolic syndrome definition did not improve CVD discrimination or risk prediction.

A number of different definitions for the metabolic syndrome have been proposed (1–4), with the cut points for each component and their combinations derived primarily from expert opinion. The lack of an empirically derived definition hinders evaluation of opposing arguments concerning the clinical utility of the metabolic syndrome.

RESEARCH DESIGN AND METHODS

Data from the prospective Framingham Offspring, Atherosclerosis Risk in Communities (ARIC), and Cardiovascular Health studies were pooled as previously described (5). Informed consent and appropriate institutional review board approval were obtained for each study. For the Framingham Offspring Study, the fourth examination was used as the baseline visit because it was the first time point at which waist circumference was measured. Individuals reporting a history of cardiovascular disease (CVD) at baseline, fasting <8 h, aged <45 years, or underweight (BMI <18.5 kg/m²) were excluded.

CVD outcomes included probable and definite fatal and nonfatal coronary heart disease (CHD) and stroke for all studies (6–9). CHD was defined as myocardial infarction, silent myocardial infarction, or CHD death, and stroke included both hemorrhagic and ischemic subtypes.

Participants were classified as having metabolic syndrome according to three existing definitions as follows. ATP III definition, three or more of the following criteria: 1) waist circumference >102 cm in men and >88 cm in women, 2) triglycerides ≥1.7 mmol/L, 3) HDL cholesterol <1.0 mmol/L (men) or <1.3 mmol/L (women), 4) blood pressure ≥130/85 mmHg or antihypertensive medication use, and 5) glucose ≥5.6 mmol/L or antidiabetes medication use (or physician diagnosis for ARIC). International Diabetes Federation (IDF) definition, abdominal adiposity (waist circumference ≥94 cm [men] or ≥80 cm [women]) and two or more risk factors from numbers 2 through 5 above. Harmonized definition, identical to the ATP III definition except that IDF waist circumference cut points were used. Information on medications for raising HDL cholesterol, needed for the Harmonized definition, were not ascertained but were expected to be infrequent given that baseline exams occurred prior to 1992.

The cut point for each component of the metabolic syndrome and the resulting number of such cut points that maximized the sum of sensitivity and specificity for incident CVD events were calculated. The relative integrated discrimination improvement (RIDI) was then calculated (10), with values greater than zero indicating better overall discrimination for empirically derived versus existing definitions. Cox proportional hazards regression was used to calculate hazard ratios (HRs) for incident CVD events associated with each metabolic syndrome definition, adjusting for age, sex, race, education level, smoking status, alcohol intake, and study (ARIC, Cardiovascular Health Study [CHS], and Framingham Offspring).

RESULTS

Across the three studies (n = 20,581), 1,783 incident CVD events occurred over 9 years of follow-up. In the overall sample, the optimal systolic and diastolic blood pressure and triglyceride cut points were lower than those in existing definitions, whereas the glucose cut point was identical (Table 1). Within demographic subgroups, the optimal cut point varied by component and subgroup.

Table 1

Empirically derived cut points and associated discrimination performance statistics for metabolic syndrome components by demographic subgroups

The presence of ≥3 components at levels above the optimal cut point (below the optimal cutpoint for HDL cholesterol), identified in Table 1, provided the highest sum of sensitivity and specificity, and two empirically derived metabolic syndrome definitions were created: 1) three or more components using cut points derived in the full sample (uniform cut point definition) and 2) three or more components using demographic subgroup-specific cut points.

More people were categorized as having metabolic syndrome using the empirically derived definitions (48.3% for uniform cut points and 51.0% for subgroup-specific cut points) compared with the ATP III (38.4%), IDF (35.6%), or Harmonized (46.0%) definitions. The empirically derived definitions, using uniform and subgroup-specific cut points, had higher sensitivity but lower specificity (0.65/0.53 and 0.67/0.50, respectively) versus the ATP III (0.53/0.63), IDF (0.51/0.66), or Harmonized (0.64/0.56) definitions. The empirically derived definition with uniform cut points provided little improvement over the ATP III (RIDI 5% [95% CI −6 to 18]), IDF (2% [−9 to 14]), or Harmonized (−0.2% [−11 to 11]) definitions. Results were markedly similar when subgroup-specific cut points were used.

All metabolic syndrome definitions were associated with an increased HR for incident CVD in multivariable-adjusted analyses (1.7 [95% CI 1.6–1.9], 1.8 [1.6–2.0], and 1.9 [1.7–2.0] for ATP III, IDF, and Harmonized, respectively, and 1.7 [1.6–1.9] and 1.8 [1.6–1.9] for the empirically derived uniform and subgroup-specific cut point definitions, respectively). Results were similar among individuals without hypertension or diabetes or when the HRs were adjusted for antihypertensive and lipid-lowering medication use. A fivefold cross-validation study resulted in identical cut points and similar performance characteristics.

CONCLUSIONS

The data-derived definitions of the metabolic syndrome resulting from maximization of the sum of sensitivity and specificity for CVD events resulted in little improvement in discrimination compared with existing definitions. Also, the HRs for CVD associated with each metabolic syndrome definition were similar.

The empirical derivation resulted in cut points similar to those based on expert opinion used in existing metabolic syndrome definitions. The exception was the empirically derived triglyceride cut point (1.4 mmol/L), which was substantially lower than that used in existing definitions (1.7 mmol/L). The lower triglyceride cut point resulted in a higher prevalence of the metabolic syndrome but no overall improvement in discrimination or meaningful difference in the HR for CVD.

For the ATP III, IDF, and Harmonized definitions, sex-specific cut points are applied for HDL cholesterol and waist circumference criteria. In the current study, subgroup-specific cut points resulted in higher sensitivity in some groups at the expense of lower specificity. In other subgroups, specificity was higher but sensitivity lower. Given this, the lack of improvement in discrimination when using subgroup-specific cut points, and the similarity of HRs between the empirically derived uniform cut point definition and existing definitions, which use sex-specific cut points, these data suggest that uniform cut points in all subgroups may be sufficient for discrimination and assessing associations. This results in a simplified clinical application.

Although measurement protocols were not identical across the three studies, findings were similar within each study (data not shown). The strength of this study comes from the use of a large sample with over 1,700 incident CVD events.

Although the ATP III, IDF, and Harmonized definition cut points were not empirically derived, the improvements in CVD risk prediction seen with empirically derived cut points were small, suggesting that the existing metabolic syndrome definitions will be sufficient for use in clinical practice and research studies should the metabolic syndrome debate conclude in a recommendation for its continued use.

Acknowledgments

This study was supported by a grant from the National Heart, Lung, and Blood Institute (NHLBI) of the National Institutes of Health (R21 HL089625 [to R.P.W.]). The CHS, ARIC, and Framingham Offspring Study are conducted and supported by the NHLBI in collaboration with the CHS, ARIC, and Framingham Offspring Study investigators, respectively.

No potential conflicts of interest relevant to this article were reported.

R.P.W. was involved in the design of the study and wrote the manuscript. A.P.M. designed data analysis programs, wrote the statistical methods section, and edited the manuscript. M.K. was involved in the design of the study and edited the manuscript. P.M. was involved in the design of the study, assisted with data analysis, and edited the manuscript. D.W. performed the statistical analysis. H.W.C. was involved in the design of the study and edited the manuscript. A.D.O. edited the manuscript. K.R. was involved in the design of the study and edited the manuscript. V.F. was involved in the design of the study and edited the manuscript.

Footnotes

This article was prepared using a limited access dataset obtained from the NHLBI and does not necessarily reflect the opinions or views of the CHS, ARIC, Framingham Offspring studies, or the NHLBI.

References

1. National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation 2002;106:3143–3421. [PubMed]

2. Grundy SM, Brewer HB, Jr, Cleeman JI, Smith SC, Jr, Lenfant C., American Heart Association. National Heart, Lung, and Blood Institute Definition of metabolic syndrome: report of the National Heart, Lung, and Blood Institute/American Heart Association conference on scientific issues related to definition. Circulation 2004;109:433–438. [PubMed]

3. International Diabetes Federation. The IDF consensus worldwide definition of the metabolic syndrome [article online], 2005. Available from http://www.idf.org/webdata/docs/Metabolic_syndrome_definition.pdf Accessed 14 August 2006.

4. Alberti KG, Eckel RH, Grundy SM, et al. International Diabetes Federation Task Force on Epidemiology and Prevention. National Heart, Lung, and Blood Institute. American Heart Association. World Heart Federation. International Atherosclerosis Society. International Association for the Study of Obesity Harmonizing the metabolic syndrome: a joint interim statement of the International Diabetes Federation Task Force on Epidemiology and Prevention; National Heart, Lung, and Blood Institute; American Heart Association; World Heart Federation; International Atherosclerosis Society; and International Association for the Study of Obesity. Circulation 2009;120:1640–1645. [PubMed]

5. Wildman RP, McGinn AP, Lin J, et al. Cardiovascular disease risk of abdominal obesity vs. metabolic abnormalities. Obesity (Silver Spring). 19 August 2010 [Epub ahead of print] [PMC free article] [PubMed]

6. Rosamond WD, Folsom AR, Chambless LE, et al. Stroke incidence and survival among middle-aged adults: 9-year follow-up of the Atherosclerosis Risk in Communities (ARIC) cohort. Stroke 1999;30:736–743. [PubMed]

7. White AD, Folsom AR, Chambless LE, et al. Community surveillance of coronary heart disease in the Atherosclerosis Risk in Communities (ARIC) Study: methods and initial two years’ experience. J Clin Epidemiol 1996;49:223–233. [PubMed]

8. Ives DG, Fitzpatrick AL, Bild DE, et al. Surveillance and ascertainment of cardiovascular events. The Cardiovascular Health Study. Ann Epidemiol 1995;5:278–285. [PubMed]

9. Abbott RD, McGee DL. The Probability of Developing Certain Cardiovascular Diseases in Eight Years at Specified Values of Some Characteristics: The Framingham Study: An Epidemiological Investigation of Cardiovascular Disease. Bethesda, MD, National Heart, Lung, and Blood Institute, 1987.

10. Pencina MJ, D’Agostino RB, Sr, D’Agostino RB, Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27:157–172. [PubMed]

Articles from Diabetes Care are provided here courtesy of
American Diabetes Association