|Home | About | Journals | Submit | Contact Us | Français|
Improved identification of subjects at high risk for development of type 2 diabetes would allow preventive interventions to be targeted toward individuals most likely to benefit. In previous research, predictive biomarkers were identified and used to develop multivariate models to assess an individual's risk of developing diabetes. Here we describe the training and validation of the PreDx™ Diabetes Risk Score (DRS) model in a clinical laboratory setting using baseline serum samples from subjects in the Inter99 cohort, a population-based primary prevention study of cardiovascular disease.
Among 6784 subjects free of diabetes at baseline, 215 subjects progressed to diabetes (converters) during five years of follow-up. A nested case-control study was performed using serum samples from 202 converters and 597 randomly selected nonconverters. Samples were randomly assigned to equally sized training and validation sets. Seven biomarkers were measured using assays developed for use in a clinical reference laboratory.
The PreDx DRS model performed better on the training set (area under the curve [AUC] = 0.837) than fasting plasma glucose alone (AUC = 0.779). When applied to the sequestered validation set, the PreDx DRS showed the same performance (AUC = 0.838), thus validating the model. This model had a better AUC than any other single measure from a fasting sample. Moreover, the model provided further risk stratification among high-risk subpopulations with impaired fasting glucose or metabolic syndrome.
The PreDx DRS provides the absolute risk of diabetes conversion in five years for subjects identified to be “at risk” using the clinical factors.
The current gold standard method used in clinical settings for identifying individuals with type 2 diabetes mellitus (T2DM) is the oral glucose tolerance test (OGTT). Even so, the OGTT is not widely used as a risk assessment tool in routine clinical practice because it is inconvenient, time-consuming, and costly.1,2 Fasting plasma glucose is more widely employed because it is more convenient and inexpensive to measure, although it does not predict diabetes onset as accurately as OGTT. An additional shortcoming of standard glucose measures as risk assessment tools is that, by the time the body is no longer able to regulate glucose normally, the disease has been progressing for several years, and complications have already appeared in a significant fraction of individuals.3 This calls into question the rationale of using a single variable that captures only one facet of disease pathology and progression to estimate risk when the risk of harm actually varies based on a wide range of variables2,4,5 and would be better assessed using a multivariable risk score.6
Various guidelines recommend the routine use of clinical risk assessment tools, and a variety of models that use clinical information and/or laboratory measurements for the prediction of future diabetes onset have been developed. However, these models have very low utilization rates because of the challenges and limitations of the OGTT and physician-calculated models.7–9 An improved method for diabetes risk assessment for subjects considered to be “at risk” based on a variety of clinical and anthropometric factors would enable more people to be evaluated and at-risk individuals to be better identified. Ideally, a new method would provide improved risk assessment in a format that facilitates use in routine clinical practice.
Developing a model for assessment of diabetes risk has been challenging because multiple metabolic pathways are dysregulated in T2DM10,11 and dysregulation may begin years or even decades before diagnosis of diabetes.12,13 In our previous study,14 we systematically analyzed multiple biomarkers measured in fasting serum samples and developed a model to predict future diabetes onset.
The objective of the current analysis was to validate this previously developed model in a clinical laboratory setting to provide an objective method for evaluating the risk of developing diabetes within five years. Baseline samples from individuals who developed diabetes within five years (“converters”) and random controls (“nonconverters”) were selected from the Inter99 cohort. Inter99 is a population-based primary prevention study of cardiovascular disease in subjects 30 to 60 years of age selected from 11 municipalities in Copenhagen County, Denmark.15 These samples were used to develop and independently validate the PreDx™ Diabetes Risk Score (DRS) in a clinical laboratory setting.
Previously, we measured 58 serum proteins representing multiple biological pathways (Figure 1) using molecular counting technology16 as well as routine laboratory measures [such as fasting plasma glucose, fasting serum insulin, triglycerides, total cholesterol, high-density lipoprotein (HDL) cholesterol, and low-density lipoprotein (LDL) cholesterol] to identify those biomarkers that were most informative for predicting the incidence of T2DM within five years. Using a variety of statistical approaches, predictive biomarkers—adiponectin, C-reactive protein, ferritin, glucose, insulin, and interleukin-2 receptor alpha—were selected for model development.14 We classified these biomarkers as involved in various pathways using Ingenuity® Systems' IPA-Biomarker™ Analysis software (http://www.ingenuity.com).
The plan for training and validation of the PreDx DRS model is shown in Figure 2. Of 6784 subjects free of diabetes at baseline who participated in the Inter99 study,15 215 subjects progressed to diabetes (converters) during five years of follow-up. A nested case-control study was performed using serum samples available from 202 converters and 597 randomly selected nonconverters. Samples were randomly assigned to a training set and a validation set comprised of roughly equal numbers of converters and nonconverters with equivalent characteristics at baseline (Table 1). The PreDx DRS logistic-regression model which comprises hemoglobin A1c (HbA1c) plus six biomarkers previously identified14 was fit to the training set (100 converters and 299 nonconverters) and performance assessed by comparing the area under the curve (AUC) of a receiver operator characteristic (ROC) curve of the model to that of fasting glucose alone. The performance of the PreDx DRS model was then validated using the validation set (102 converters and 298 nonconverters) without refitting coefficients.
Anthropometric measurements (e.g., body mass index [BMI], blood pressure, and waist), routine laboratory measures (e.g., triglycerides and cholesterol), and the OGTT were performed as described.14 Samples were collected following a 10 h fast and serum stored at -19 °C until analyses. Serum biomarkers were measured using assays developed for use in a clinical laboratory as indicated in Table 2.
A noninvasive clinical model was developed using the same approach as that used for biomarker selection and model development.14 Of the noninvasive clinical measures available in the current data set, the subset identified as most informative included age, gender, BMI, waist circumference, and family history. A logistic regression model incorporating these clinical measures was fit on the training set and then applied to the validation set without refitting coefficients.
Performance of the PreDx DRS was also assessed in three subpopulations of the Inter99 cohort at higher risk for progressing to T2DM within five years, defined by age (>39 years), BMI (≥25 kg/m2), impaired fasting glucose (IFG) (serum glucose 100–125 mg/dl),19 and metabolic syndrome.20 A subject was considered to have metabolic syndrome if three of five criteria were met: waist circumference >35 in. for women or >40 in. for men, triglycerides >150 mg/dl, HDL <50 mg/dl for women or <40 mg/dl for men, blood pressure >130/85 (if either systolic or diastolic pressure is elevated, it is considered a risk), or fasting plasma glucose >100 mg/dl.20 For each of these subpopulations, results were adjusted for five-year incidence of diabetes in Inter99 using Bayes' Rule.
The performance of the PreDx DRS model in estimating the five-year risk of T2DM was assessed by ROC curve analysis (Figure 3). When fitted to the training set, the PreDx DRS model yielded an AUC of 0.837. By comparison, fasting plasma glucose applied to the training set yielded an AUC of 0.779. These results demonstrate that the PreDx DRS model is statistically significantly better than fasting plasma glucose in assessing the five-year risk of T2DM (p = .0003).
The PreDx DRS model was then applied to the sequestered validation set without refitting model coefficients. When applied to the validation set, the PreDx DRS model yielded an AUC of 0.838. The similarity of the AUCs obtained for the PreDx DRS model on the training and validation sets validates model performance, suggesting the model is not over-fit. As shown in Table 3, the PreDx DRS ranges from 0 to 10, and median and mean scores for converters were statistically different from scores for nonconverters in both the training and the validation sets.
Comparison of AUC values in the validation set also shows that the PreDx DRS model discriminates between converters and nonconverters better than any other clinical measure tested, as illustrated in Figure 4: HbA1c (p < .0001), fasting insulin (p = .0017), HOMA-IR (p = .0039), fasting plasma glucose (p = .0003), and a model derived from noninvasive clinical measures (p = .0069).
The PreDx DRS model provides a continuous measure of risk of progressing to T2DM within five years. As shown in Figure 5, observed risk is very close to predicted risk, suggesting that that the model is well calibrated. For simplicity, we have defined three risk strata, with low risk defined as PreDx DRS <4.5, moderate risk as PreDx DRS ≥4.5 and <8.0, and high risk as PreDx DRS ≥8.0.
The superior risk stratification of the PreDx DRS also is observed within specific higher-risk subpopulations. For example, fasting glucose status gives only limited stratification among higher-risk individuals >39 years with a BMI ≥25 kg/m2, a subpopulation whose average risk of progressing to T2DM within five years is 5.7%.
By comparison, those with IFG (51% of subpopulation) have a 9% risk, and those with normal fasting glucose (49% of subpopulation) have a 2% risk (Figure 6A). When this same subpopulation is stratified by PreDx DRS, 24% of those classified as high risk (11% of subpopulation) converted to T2DM within five years, while conversion occurred in only 1% of those classified as low risk (54% of subpopulation) (Figure 6B). Hence the PreDx DRS enabled identification of individuals whose actual risk of progressing to T2DM was more than four times higher or four times lower than the average risk (5.7%) for this subpopulation.
PreDx DRS also successfully stratified even higher-risk subpopulations, such as individuals >39 years with a BMI ≥25 kg/m2 that also have IFG (Figure 6C). As compared to the 9% average risk in the IFG subpopulation, the PreDx DRS identified a subset with a 24% risk (21% of IFG subgroup) as well as a subset with a 2% risk (26% of IFG subgroup) of progressing to T2DM within five years. Similarly, the PreDx DRS provides further stratification of risk among individuals with metabolic syndrome who are considered at high risk for T2DM based on a combination of clinical measurements such as waist circumference, triglyceride levels, HDL levels, blood pressure, and/or fasting glucose (Figure 6D). As compared to the 8.3% average risk in the metabolic syndrome subpopulation, the PreDx DRS identified a subset with a 23% risk (23% of metabolic syndrome subgroup) and a subset with a 2% risk (23% of metabolic syndrome subgroup) of progressing to T2DM within five years.
There is great clinical need for a robust and convenient tool for identifying individuals at highest risk of developing T2DM so that clinicians can implement an effective diabetes prevention program that may delay or prevent development of the disease. In this study, we validated a DRS model that provides a quantitative estimate of the risk of progressing to T2DM within five years. The performance of the PreDx DRS model was very similar in the training and validation sets, suggesting that the model is not over-fit and may be generalized to other populations. Moreover, the performance of the PreDx DRS model was superior to other clinical measures in identifying a high-risk subpopulation—this model performed better in assessing T2DM risk than fasting plasma glucose, fasting insulin, HOMA-IR, HbA1c, and a model derived from noninvasive clinical measures.
We developed the PreDx DRS using biomarkers representing multiple pathways implicated in diabetes development, including biomarkers involved in inflammatory response, fat and carbohydrate metabolism, coagulation, metabolic disorders, and cell death. The PreDx DRS model validated in this study utilized assays developed and standardized for use in the clinical laboratory and had very similar performance to the model derived on a research platform.14 It will be important to test multiethnic cohorts to further verify the performance of the PreDx DRS.
The PreDx DRS is a convenient alternative for assessing T2DM risk in a clinical setting. It requires a simple blood draw, which is easily incorporated into routine clinical practice, with testing performed by a Clinical Laboratory Improvement Amendments (CLIA)-certified laboratory (Tethys Clinical Laboratory, Emeryville, CA). A single estimate of absolute five-year risk is then provided to the physician. The PreDx DRS may be used along with other clinical information to identify patients at highest risk for progressing to diabetes—possibly before irreversible organ damage occurs. Early detection of these high-risk individuals could allow for optimal patient management by enabling clinicians to focus appropriate resources earlier to implement a diabetes prevention program.
This study validated the performance of the PreDx DRS and demonstrated that it was better than any other clinical measure tested in estimating the risk of progressing to T2DM within five years. In current clinical practice, both IFG and metabolic syndrome are used to identify subjects who are “at risk” for the development of T2DM. It is of major clinical relevance and significance that the PreDx DRS provides additional information to further stratify risk within these subgroups. By providing a more accurate and convenient measure of T2DM risk, the PreDx DRS may be considered a valuable tool for physicians to use in conjunction with clinical factors to identify individuals with the highest risk of developing diabetes in five years and for whom prevention programs may reduce the serious health consequences of T2DM.
The Inter99 study was initiated by T. Jørgensen (principal investigator), K. Borch-Johnsen (principal investigator for the diabetes part), T. Thomsen, and H. Ibsen. The present steering group comprises T. Jørgensen (principal investigator), K. Borch-Johnsen (co-principal investigator), and C. Pisinger. The authors gratefully acknowledge the skillful technical work of Amalia Ocampo and Claudia Elkin, Tethys Bioscience, and thank Anthony Sponzilli for graphic illustrations and Linda Wuestehube for editorial assistance.