PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jdstLink to Publisher's site
 
J Diabetes Sci Technol. 2010 January; 4(1): 111–118.
Published online 2010 January. doi:  10.1177/193229681000400114
PMCID: PMC2825631

Accuracy of a New Real-Time Continuous Glucose Monitoring Algorithm

Abstract

Background

Through minimally invasive sensor-based continuous glucose monitoring (CGM), individuals can manage their blood glucose (BG) levels more aggressively, thereby improving their hemoglobin A1c level, while reducing the risk of hypoglycemia. Tighter glycemic control through CGM, however, requires an accurate glucose sensor and calibration algorithm with increased performance at lower BG levels.

Methods

Sensor and BG measurements for 72 adult and adolescent subjects were obtained during the course of a 26-week multicenter study evaluating the efficacy of the Paradigm® REAL-Time (PRT) sensor-augmented pump system (Medtronic Diabetes, Northridge, CA) in an outpatient setting. Subjects in the study arm performed at least four daily finger stick measurements. A retrospective analysis of the data set was performed to evaluate a new calibration algorithm utilized in the Paradigm® Veo™ insulin pump (Medtronic Diabetes) and to compare these results to performance metrics calculated for the PRT.

Results

A total of N = 7193 PRT sensor downloads for 3 days of use, as well as 90,472 temporally and nonuniformly paired data points (sensor and meter values), were evaluated, with 5841 hypoglycemic and 15,851 hyperglycemic events detected through finger stick measurements. The Veo calibration algorithm decreased the overall mean absolute relative difference by greater than 0.25 to 15.89%, with hypoglycemia sensitivity increased from 54.9% in the PRT to 82.3% in the Veo (90.5% with predictive alerts); however, hyperglycemia sensitivity was decreased only marginally from 86% in the PRT to 81.7% in the Veo.

Conclusions

The Veo calibration algorithm, with sensor error reduced significantly in the 40- to 120-mg/dl range, improves hypoglycemia detection, while retaining accuracy at high glucose levels.

Keywords: continuous glucose monitoring, hypoglycemia, insulin pump, sensor

Introduction

The landmark Diabetes Control and Complications Trial (DCCT)1,2 clearly demonstrated the benefits of treating to a target hemoglobin A1c (HbA1c) of 7.0% or less after reporting a vast decrease in morbidity and a slowed onset, or progression, of severe complications in a type 1 diabetes population. This outcome was attributed to reduced blood glucose (BG) levels through the use of intensive insulin therapy, which required self-monitoring of blood glucose (SMBG) levels at least four times a day.

However, fear of hypoglycemia (FoH) deters insulin-dependent patients from more aggressive management of their BG levels. In fact, before home blood glucose monitoring, physicians encouraged patients to aim for a higher-than-normal BG level—a phenomenon that has been reviewed extensively by Wild and colleagues.3 In contrast, it has been demonstrated that patients who perform only SMBG with a customary BG meter with four daily finger stick measurements could miss up to 71% of hypoglycemic events, whereas testing up to seven times daily could miss 58% of events when compared to continuous glucose monitoring (CGM).4

Additionally, the ability of CGM to reduce HbA1c has been demonstrated, at least under clinical supervision.5,6 The introduction of glucose sensors, therefore, has provided evidence indicating that CGM can indeed further reduce HbA1c levels and glycemic variability, with increased awareness to hypoglycemia, when compared to standard SMBG. Also, because the goal of intensive therapy diabetes management is to reduce HbA1c levels without increasing the incidence of hypoglycemia, then it is evident that CGM therapy can be very beneficial.

This article focuses on the accuracy of CGM—specifically the accuracy of a new calibration algorithm. The motivation for improved accuracy is to enhance further the efficacy of CGM in lowering HbA1c, as already seen in several clinical studies. To date, the largest of these studies included 322 patients in a multicenter randomized control trial sponsored by the Juvenile Diabetes Research Foundation, which compared the efficacy of CGM to that of SMBG.7 Overall, results demonstrated that CGM significantly improves glycemia over standard SMBG with a reduction of 0.5% in HbA1c, although results were not statistically significant in the adolescent population, likely due to poor adherence. Similarly, the European GuardControl Trial8 evaluated the efficacy of CGM in a multicenter study that included 156 patients wearing the Guardian® RT monitor (Medtronic Diabetes, Northridge, CA). The outcome showed that subjects in the treatment arm decreased their HbA1c levels by an average of 1%—a 0.6% improvement over the control group using traditional SMBG—and HbA1c levels were reduced by 2% in over 26% of subjects.

In 2006, the first sensor-augmented insulin pump, Medtronic's Paradigm® REAL-Time (PRT) system, was launched, introducing the first platform with all of the required components communicating together, necessary for an ambulatory closed-loop artificial pancreas product.9 The system is composed of a Paradigm® 522/722 insulin pump enhanced with the real-time CGM component, which utilizes the same calibration algorithm as the Guardian RT, Paradigm REAL-Time Revel™ system, and Guardian REAL-Time™ system. An early prototype of this system was tested in 20 type 1 diabetes subjects who wore the device for up to 2 years.10 Study results demonstrated that participants achieved a mean reduction in HbA1c of 1.1% and that those subjects with levels greater than 7% had a 0.67 probability of reducing their level below this value in the first 3 months of use.

The STAR 1 study, sponsored by Medtronic Diabetes and conducted across seven centers, investigated the efficacy of the PRT system.11 The primary objective of this study was to demonstrate a greater reduction of HbA1c levels in subjects placed on the PRT system. In the control group, the overall change in HbA1c from baseline was –0.58 ± 0.73%, whereas the change was –0.72 ± 0.69% in the PRT group, representing a 7% decrease for the control group and an 8.3% decrease for the PRT group. Compared to the 8.2% average HbA1c outcome from 30 university hospital centers12 and the 8.0% DCCT HbA1c results,13 subjects in the study achieved an average HbA1c of 7.68%. Final results revealed that 38% of patients in the CGM sensor-augmented group reached the 7% (or below) HbA1c target versus 19% in the control insulin pump group. Additionally, the PRT group showed a significant decrease in hypoglycemia area under the curve when compared to the control group, and the PRT group also included a higher percentage of adult and adolescent subjects reaching a target HbA1c value of less than 7.0%. This development is clinically meaningful because previous attempts to reduce HbA1c levels below the 7% mark in an adolescent population have been unsuccessful due to an increased risk for hypoglycemia.14,15 Furthermore, because the control group was not able to achieve this reduction, the lowered HbA1c levels attained by adults and adolescents in the PRT group were, in all probability, due to sensor usage.

From the previously detailed studies, it is evident that through CGM therapy patients can more effectively manage their BG levels and achieve an improvement in HbA1c, while reducing the risk of hypoglycemia and, possibly, anxiety associated with FoH. However, it is also evident that improved control through CGM requires a reliable and accurate glucose sensor with an effective calibration algorithm coupled with patient compliance.

Current commercially available CGM devices are progressively demonstrating greater accuracies.16,17 In a multicenter study that included 58 subjects to evaluate the performance of the FreeStyle Navigator® (Abbott Diabetes Care, Alameda, CA), a mean absolute relative difference (MARD) of 12.8% was achieved over 5 days of sensor use with 80% sensitivity to hypoglycemia (<70 mg/dl). In a more recent study that included 72 insulin-dependent diabetic subjects wearing the SEVEN® system (DexCom, San Diego, CA), accuracy was evaluated over 7 days of sensor use, calibrating with capillary finger stick measurements. An overall MARD of 16.7% was achieved with 13.3% on day 7, which demonstrates that the DexCom 7-day system shows a significant improvement over the 3-day system, a development that is credited to new algorithm enhancements.

Similarly, this investigation evaluates the performance of a new glucose sensor calibration algorithm, now utilized in the Paradigm Veo™ insulin pump recently launched in Europe. The Veo calibration algorithm with enhancements is compared to that of the PRT by retrospective analysis of data collected during the STAR 1 clinical trial, with a range of performance metrics calculated for each system.

Methods

In the previously discussed STAR 1 trial—a randomized multicenter treat-to-target 6-month study in which continuous subcutaneous insulin infusion therapy, augmented with real-time CGM, was compared with standard SMBG and insulin pump therapy11—136 patients were enrolled across seven study centers. Ninety-six subjects were adults, of whom 57% were female, while 60% of the adolescent subjects were female.

Patients were randomized into either a control arm wearing the Paradigm 715 insulin pump or a treatment group wearing the PRT sensor-augmented insulin pump. The primary end point of HbA1c reduction over a 26-week period was the healthy glycemic control goal for diabetes patients of below 7%. Subjects had to be insulin pump users for a period greater than 6 months and have HbA1c levels greater than or equal to 7.5%. Subjects in the study group were required to perform at least four finger stick measurements daily, uploading with the Paradigm Link® blood glucose monitor (Medtronic Diabetes) for the duration of the study. Subjects wore two sensors per week and tested HbA1c five times during the study.

Insulin pump and glucose sensor data were uploaded every 2 weeks over the Internet via the CareLink Clinical® application (Medtronic Diabetes). This investigation utilized BG and raw glucose sensor data acquired from the CGM arm of the study. The accuracy of two calibration algorithms was compared by a retrospective analysis of raw data. As the difference between calibration algorithms is purely mathematical, in that each algorithm uses the same data points with the same timing, a true point-to-point analysis can be performed between the two routines. Consequently, the PRT algorithm is applied retrospectively to raw data to ensure this consistency. As the commercial PRT algorithm does not have the benefit of predictive alerts, this retrospective analysis uses predictive alerts that are part of the Paradigm REAL-Time Revel system and Guardian REAL-Time system—devices not yet launched in the United States. However, all three devices have the same calibration algorithm. The predictive alerts utilize a Savitzky–Golay finite impulse response derivative filter to estimate the sensor glucose rate of change, which is multiplied by a prediction horizon of 5–30 minutes.

Sensors were worn for up to 3 days of use by 72 subjects in the study arm of the STAR 1 trial. A total of 7193 sensor downloads provided 90,472 paired BG and sensor measurements for retrospective analysis. Performance metrics were calculated for each glucose sensor calibration algorithm based on guidelines set forth by the International Organization for Standardization18 and the Clinical and Laboratory Standards Institute (CLSI)19 for continuous interstitial fluid glucose monitoring. Sensitivity and specificity were calculated for hypo- and hyperglycemia, defined as a single BG measurement below 70 mg/dl and above 240 mg/dl, respectively. Additionally, consensus20 and Clarke21 error grid analyses were performed for each algorithm.

Results

The performance of each calibration algorithm is presented in Tables 16 for the range of metrics described previously. The PRT calibration algorithm is represented as bold following results of the Veo calibration algorithm. The aggregated error is shown in Table 1, with the error stratified by range in Table 2. The Veo algorithm outperforms the PRT algorithm, with the overall mean ± SD decreased by 0.25 ± 0.6 to 15.89 ± 16.89%, with a reduction in median error of 0.1 to 11.56%. The error is reduced further by 5.3 to 19.5% and 0.4 to 17.31% in the 40- to 80- and 80- to 120-mg/dl ranges. A slight error increase exists in the mid to upper ranges, with a 0.7% increase from 120 to 240 and 0.6% above this threshold.

Table 1.
Aggregated Errora
Table 2.
Error Stratified by Rangea
Table 6.
Clarke Error Grid Analysisa

Table 3 provides the number of points within 20 and 30% of their respective paired BG values, or 20 mg/dl for BG values below 80 mg/dl. For the Veo algorithm, more than 8% additional points reside within both boundaries in the 40- to 80-mg/dl range—with an insignificant difference between the two algorithms in all other ranges—and the number of points processed by this new algorithm decreases at higher ranges.

Table 3.
Number of Points within 20 and 30% of Their Respective Paired BG Valuesaa

The number of observed hypo- and hyperglycemic events, based on criteria for home-monitoring use, is reported in Table 4. The Veo algorithm detected 82.28% of the 5841 hypoglycemic episodes and 81.67% of the 15,851 hyperglycemic episodes. In contrast, the PRT algorithm detected 54.89 and 86.26% of all hypo- and hyperglycemic events, respectively. The new algorithm had a slightly lower specificity (96.36–98.32%) for hypoglycemia and a comparable specificity (~98%) for hyperglycemia. Sensitivity and specificity analyses for the Veo and Revel algorithms are illustrated with pie charts in Figures 1 and 22, respectively, for hypo- and hyperglycemic events. The pie charts are color coded to specify the percentage of events falling within particular limits. An additional category—accurate glucose—shows the percentage of events that fall within the error tolerance of the measuring reference device, which, in this case, is a home-monitoring BG meter. This error boundary for hypoglycemia is 70–85 mg/dl and 20% for hyperglycemia based on CLSI guidelines.17 The charts illustrate the sensitivities reported previously, including the percentage of both threshold and predicted alerts for a 15-minute prediction horizon. Likewise, the specificities are shown for true and false threshold alerts. As the PRT algorithm does not include predictive alerts, but has the same calibration algorithm as the Revel and Guardian devices, its performance can be derived from Figure 2 by only including threshold alerts in the calculations.

Table 4.
Number of Observed Hypo- and Hyperglycemic Events Based on Criteria for Home-Monitoring Usea
Figure 1.
Veo algorithm sensitivity and specificity analyses for hypoand hyperglycemic events.
Figure 2.
Revel and Guardian REAL-Time algorithm sensitivity and specificity analyses for hypo- and hyperglycemic events.

In the following alarm categories—threshold, threshold and projected, projected, no alarm accurate glucose, and false negative—when applying a 30-minute prediction horizon for hypoglycemia detection, the Veo algorithm generated accuracies of 2.62, 79.7, 11.4, 4.02, and 2.29%, respectively. Applying the same 30-minute prediction horizon to Guardian and Revel algorithms produced accuracies of 7.29, 47.6, 20.1, 13.8, and 11.3%, respectively.

Over 14 hours of data is illustrated in Figure 3 for rates of change in excess of 1 mg/dl/min, and greater rates on the decline. The 330-mg/dl calibration sample before hour 28 resulted in the PRT algorithm overreading proceeding low glucose levels and consequently failing to detect the 59-mg/dl (PRT = 81 mg/dl) hypoglycemic event at hour 34, whereas the Veo algorithm detected the event several hours before the meter indicated a hypoglycemic episode.

Figure 3.
Sensor tracings for PRT and Veo calibration algorithms.

Clarke and consensus error grid analyses are presented in Tables 5 and 66. Results are comparable—with greater than 97% of all readings in the A + B zones of the consensus error grid for both algorithms and in all ranges, with the exception of the PRT algorithm in the 40- to 80-mg/dl range. In this range, the new algorithm shows a 4% improvement. No points reside within the E zone of the consensus error grid for either algorithm. Similarly, results are comparable throughout most ranges for Clarke error grid analysis, with the exception of low BG levels in the 40- to 80-mg/dl range. In this instance, the new algorithm has more than a 13% improvement with 89.81% of paired points in the A + B zones. Both algorithms have approximately 96% of all values in the clinically accurate and benign (A + B) zones for the complete range. Overall, there are only 0.18 and 0.12% points in the E zone of the Clarke error grid for Veo and PRT algorithms, respectively, which could lead to erroneous treatment of hypo- or hyperglycemia.

Table 5.
Consensus Error Grid Analysisa

Discussion

The Paradigm Veo calibration algorithm is more accurate in the 40- to 80- and 81- to 120-mg/dl ranges, and it consequently detected significantly more hypoglycemic events. Sensitivity analysis of hypoglycemic events produced accuracies of 82.3% compared to the 54.9% detected by the PRT algorithm. Predictive alerts, with a 30-minute prediction horizon, detected 94% of all hypoglycemic events. It would, thus, be expected that an increase in sensitivity in the lower glucose ranges would result in a balanced decrease in accuracy at higher glucose ranges. However, accuracy at higher glucose ranges was affected only marginally—an expected development, as the new algorithm attempts to produce an even error distribution across the dynamic range of the sensor, reflective of the insignificant sensitivity difference for hypo- and hyperglycemia. Increasing accuracy at low blood glucose levels, therefore, while maintaining accuracy at high glucose levels, essentially broadens the dynamic range of the glucose sensor system; this is evident in Figure 3.

The Veo algorithm is more sensitive to fast glucose excursions, with a decreased filter delay (≈5 minutes), reducing the overall signal processing delay to 3.5 minutes and providing a much faster response time to rapid glucose excursions. The range of intrinsic delays for other CGM devices currently on the market was reported previously by the authors.22

It should be noted that STAR 1 data used in this investigation were acquired in an outpatient setting and evaluated retrospectively using standards for home monitoring. In this setting, accuracies are generally lower as error measurements are generally made at the maximum time duration following calibration. In contrast, evaluating sensor performance from frequent sampled in-clinic studies with laboratory reference measurements taken every 15 minutes, the sensor error is generally 3–4% lower with more accurate samples acquired by professionals.

Conclusion

Fear of hypoglycemia is a common emotion experienced by people with type 1 diabetes, often discouraging a greater intensive management of their BG levels in order to achieve better glycemic control. Features available with the Paradigm Veo, such as alerts and the low glucose suspend function, can help mitigate serious hypoglycemia, where insulin delivery is suspended beyond a predetermined threshold. While most CGM devices provide threshold and predictive alarms to warn the user of impending or occurring hypo- or hyperglycemic episodes, widespread adoption of these features is often hampered by low accuracy and false alerts. Therefore, increased accuracy, specifically at low blood glucose levels, is advantageous.

The Paradigm Veo calibration algorithm demonstrates performance improvement over the current PRT system, with accuracy improved in nearly all performance categories. Hypoglycemia sensitivity is increased by greater than 50%, with 90.5 and 94% of all hypoglycemic events detected for prediction horizons of 15 and 30 minutes, respectively. The mean error in the low glucose range is decreased by approximately 5%, and the sensor signal is more reactive to fast glucose excursions. Furthermore, sensor signal processing delays are reduced considerably in the Veo algorithm. This reduction in time delay provides earlier and more accurate alerts, where hypoglycemia can be detected and predicted sooner, thereby enhancing patient safety and affording more time to self-treat potential reactions.

Abbreviations

BG
blood glucose
CLSI
Clinical and Laboratory Standards Institute
CGM
continuous glucose monitoring
DCCT
Diabetes Control and Complications Trial
FoH
fear of hypoglycemia
HbA1c
hemoglobain A1c
MARD
mean absolute relative difference
PRT
Paradigm REAL-Time
SMBG
self-monitoring of blood glucose

References

1. The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329(14):977–986. [PubMed]
2. The Diabetes Control and Complications Trial Research Group. Implementation of treatment protocols in the Diabetes Control and Complications Trial. Diabetes Care. 1995;18(3):361–376. [PubMed]
3. Wild D, von Maltzahn R, Brohan E, Christensen T, Clauson P, Gonder-Frederick L. A critical review of the literature on fear of hypoglycemia in diabetes: implications for diabetes management and patient education. Patient Educ Couns. 2007;68(1):10–15. [PubMed]
4. Jungheim K, Wientjes KJ, Heinemann L, Lodwig V, Koschinsky T, Schoonen AJ. Glucose Monitoring Study Group. Subcutaneous continuous glucose monitoring: feasibility of a new microdialysis-based glucose sensor system. Diabetes Care. 2001;24(9):1696–1697. [PubMed]
5. Bode BW, Gross TM, Thornton KR, Mastrototaro JJ. Continuous glucose monitoring used to adjust diabetes therapy improves glycosylated hemogloba pilot study. Diabetes Res Clin Pract. 1999;46(3):183–189. [PubMed]
6. Gross TM, Bode BW, Einhorn D, Kayne DM, Reed JH, White NH, Mastrototaro JJ. Performance evaluation of the MiniMed continuous glucose monitoring system during patient home use. Diabetes Technol Ther. 2000;2(1):49–56. [PubMed]
7. Tamborlane WV, Beck RW, Bode BW, Buckingham B, Chase HP, Clemons R, Fiallo-Scharer R, Fox LA, Gilliam LK, Hirsch IB, Huang ES, Kollman C, Kowalski AJ, Laffel L, Lawrence JM, Lee J, Mauras N, O'Grady M, Ruedy KJ, Tansey M, Tsalikian E, Weinzimer S, Wilson DM, Wolpert H, Wysocki T, Xing D. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Continuous glucose monitoring and intensive treatment of type 1 diabetes. N Engl J Med. 2008;359(14):1464–1476. [PubMed]
8. Deiss D, Bolinder J, Riveline JP, Battelino T, Bosi E, Tubiana-Rufi N, Kerr D, Phillip M. Improved glycemic control in poorly controlled patients with type 1 diabetes using real time continuous glucose monitoring. Diabetes Care. 2006;29(12):2730–2732. [PubMed]
9. Hovorka R, Wilinska ME, Chassin LJ, Acerini CL, Dunger DB. The artificial pancreas: making headway. Practical Diabetes Int. 2007;24(2):56–58.
10. Mastrototaro JJ, Cooper KW, Soundararajan G, Sander JB, Shah RV. Clinical experience with an integrated continuous glucose sensor/insulin pump platform: a feasibility study. Adv Ther. 2006;23(5):725–732. [PubMed]
11. Hirsch IB, Abelseth J, Bode BW, Fischer JS, Kaufman FR, Mastrototaro J, Parkin CG, Wolpert HA, Buckingham BA. Sensor-augmented insulin pump therapy: results from the first randomized treat-to-target study. Diabetes Technol Ther. 2008;10(5):377–383. [PubMed]
12. Writing Team for the Diabetes Control and Complications Trial/ Epidemiology of Diabetes Interventions and Complications Research Group. Sustained effect of intensive treatment of type 1 diabetes mellitus on development and progression of diabetic nephropathy. JAMA. 2003;290(16):2159–2167. [PMC free article] [PubMed]
13. Allen C, Zaccaro DJ, Palta M, Klein R, Duck SC, D'Alessio DJ. Glycemic control in early IDDM. The Wisconsin Diabetes Registry. Diabetes Care. 1992;15(8):980–987. [PubMed]
14. Amiel SA, Sherwin RS, Simonson DC, Lauritano AA, Tamborlane WV. Impaired insulin action in puberty. A contributing factor to poor glycemic control in adolescents with diabetes. N Engl J Med. 1986;315(4):215–219. [PubMed]
15. Grant RW, Buse JB, Meigs JB. University HealthSystem Consortium (UHC) Diabetes Benchmarking Project Team. Quality of diabetes care in U.S. academic medical centers. Diabetes Care. 2005;28(2):337–442. [PMC free article] [PubMed]
16. Weinstein RL, Schwartz SL, Brazg RL, Bugler JR, Peyser TA, McGarraugh GV. Accuracy of the 5-day FreeStyle Navigator Continuous Glucose Monitoring System: comparison with frequent laboratory reference measurements. Diabetes Care. 2007;30(5):1125–1130. [PubMed]
17. Zisser H, Bailey TS, Schwartz S, Ratner RE, Wise J. Accuracy of the SEVEN continuous glucose monitoring system: comparison with frequently sampled venous glucose measurements. J Diabetes Sci Technol. 2009;3(5):1146–1154. [PMC free article] [PubMed]
18. International Organization for Standardization: ISO 15197. International Organization for Standardization (ISO) Geneva; 2003. In vitro diagnostic test systems—requirements for blood glucose monitoring systems for self testing in managing diabetes mellitus.
19. Clinical and Laboratory Standards Institute. Clinical and Laboratory Standards Institute document POCT05-A. Wayne: Clinical and Laboratory Standards Institute; 2008. Performance metrics for continuous interstitial glucose monitoring; approved guideline.
20. Parkes JL, Slatin SL, Pardo S, Ginsberg BH. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care. 2000;23(8):1143–1148. [PubMed]
21. Clarke WL, Cox D, Gonder-Frederick LA, Carter W, Pohl SL. Evaluating clinical accuracy of systems for self-monitoring of blood glucose. Diabetes Care. 1987;10(5):622–628. [PubMed]
22. Keenan DB, Voskanyan G, Mastrototaro JJ, Steil GM. Delays in minimally invasive continuous glucose monitoring devices: a review of current technology. J Diabetes Sci Technol. 2009;3(5):1207–1214. [PMC free article] [PubMed]

Articles from Journal of Diabetes Science and Technology are provided here courtesy of Diabetes Technology Society