PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jdstLink to Publisher's site
 
J Diabetes Sci Technol. 2010 January; 4(1): 119–122.
Published online 2010 January.
PMCID: PMC2825632

The Accuracy of a New Real-Time Continuous Glucose Monitoring Algorithm: An Analysis

Boris Kovatchev, Ph.D. and Marc Breton, Ph.D.

Abstract

In this issue of Journal of Diabetes Science and Technology, Keenan and colleagues used archival data from the STAR 1 clinical trial (Medtronic Diabetes) to support the claim that the new Veo™ calibration algorithm improves the accuracy of continuous glucose monitoring, particularly in the critical hypoglycemic range. Extensive data analyses are presented to support this claim; the results are convincing, and the estimated improvement in hypoglycemic detection from 55% for the standard calibration to 82% for the Veo is particularly impressive. We can therefore conclude that the Veo algorithm has the potential to improve the accuracy of hypoglycemia alarms and ultimately contribute to closed-loop control. However, the presented results should be interpreted cautiously because they are based on retrospective analysis and are heavily dependent on the distribution of blood glucose levels observed in a particular data set.

Keywords: continuous glucose monitoring, continuous glucose monitoring accuracy, hypoglycemia

Studies have documented the benefits of continuous glucose monitoring (CGM)13 and charted guidelines for CGM clinical use4,5 and its future as a precursor to closed-loop control.6 However, while CGM has the potential to revolutionize the control of diabetes, it also generates data streams that are both voluminous and complex. The utilization of such data requires an understanding of the physical, biochemical, and mathematical principles and properties involved in this technology. It is important to know that CGM devices measure glucose concentration in a different compartment: the interstitium. Interstitial glucose (IG) fluctuations are related to blood glucose (BG) presumably via diffusion process.7,8 To account for the gradient between BG and IG, CGM devices are calibrated with capillary glucose, which brings the typically lower IG concentration to BG levels. Successful calibration would adjust the amplitude of IG fluctuations with respect to BG. Calibration quality, however, is influenced by the possible time lag due to BG-to-IG transport and the sensor lag time (instrument delay, primarily due to data smoothing). Because such a time lag could greatly influence the accuracy of CGM, a number of studies were dedicated to its investigation, yielding various results.912 In most studies, IG lagged behind BG (most of the time) by 4–10 min, regardless of the direction of BG change; however, negative time lag was reported as well. The formulation of the push–pull phenomenon offered reconciliation of these results and provided arguments for a more complex BG–IG relationship than a simple constant or directional time lag.11,12 In addition, errors from loss of sensitivity and random noise confound CGM data.13 Thus, while the accuracy of CGM is increasing, it is still below the accuracy of direct BG measurement.1418 Thus calibration algorithms capable of reducing CGM error are important additions to the arsenal of data processing techniques that help transfer raw current into BG readings. Algorithms reducing the time lag are critically important as well.

In an article entitled “The Accuracy of a New Real-Time Continuous Glucose Monitoring Algorithm” in this issue of Journal of Diabetes Science and Technology, Keenan and colleagues19 evaluate the accuracy of a new calibration algorithm used in the Paradigm® Veo™ insulin pump (Medtronic Diabetes) and compare the results to the standard Paradigm REAL-Time (PRT) calibration method. Archival data from the STAR 1 clinical trial comparing insulin pump therapy augmented with real-time CGM versus standard self-monitoring of blood glucose (SMBG) were used to support the claim that the Veo algorithm improves CGM accuracy, particularly in the critical hypoglycemic range. In order to ensure a fair comparison, the authors go back to raw current and perform both the Veo and the standard PRT calibrations using the same SMBG data points. The data set is reasonably large, containing data for 72 subjects in the active CGM study arm for a total of 90,472 CGM–SMBG data pairs.

None of the calibration algorithms is described in the article by Keenan and colleagues, and it is therefore unclear whether the Veo algorithm has any analytical advantages over the standard PRT calibration. From Figure 3, we can speculate that the Veo algorithm uses a calibration function with a steeper slope at low BG levels than the standard calibration; for example, using exponential versus linear type of calibration would have that effect. In the absence of a formula, however, we can only rely on the data to compare the performance of one “black box” versus another. This being the case, the performance evaluation becomes heavily dependent on the distribution of the data used for comparison. One example would clarify this statement:

Overall, the Veo algorithm had a mean absolute relative difference (MARD) of 15.89% versus 16.14% for the standard PRT calibration, i.e., Veo outperformed PRT by 0.25%. This overall accuracy was achieved by data that had distribution presented in Table 1 (line 1).19

Table 1.
Dependence of Algorithm Accuracy on the Distribution of the Self-Monitoring of Blood Glucose Data

If we assume the hypothetical distribution presented in Table 1 (line 2), which is weighted toward hyperglycemia, then the MARD results become exactly the opposite: the standard PRT algorithm outperforms Veo by 0.25%. Thus, without analytical knowledge of the calibration formulas, no statements about overall accuracy could be made, because the overall accuracy can be easily biased by the distribution of the data in hand. The Clinical and Laboratory Standards Institute (CLSI) Guidelines for evaluating the accuracy of CGMs emphasize the importance of the distribution of the test data and suggest desirable distribution characteristics.20This is, of course, not the case with the algorithm accuracy within each of the BG regions in Table 1: the performance of the Veo algorithm in the hypoglycemic region appears better than the performance of the standard PRT calibration. Statistical significance data are not presented; thus it remains unclear whether this result is not due to chance. A small penalty in accuracy is paid at the high BG range, which is to be expected, because calibration typically uses parameterized curves prone to certain inertia, i.e., if one end of the curve is overfitted, the other would naturally become suboptimal.

Extensive analyses aim to confirm the higher accuracy of the Veo algorithm in the hypoglycemic range. Particularly impressive is the increase in hypoglycemia sensitivity from 54.9% in the PRT to 82.3% in the Veo. Consensus and Clarke error grid analyses are used to additionally highlight the clinical importance of the observed accuracy differences. Unfortunately, because the reference SMBG readings are not frequent enough and not equally spaced in time, these data do not allow the use of the continuous glucose error grid analysis (CGEGA)17 or any other analysis of trend accuracy. Perhaps future studies could remedy this deficiency and would take full advantage of the CGEGA, which is designed on the premise that CGM accuracy should be evaluated differently in different BG regions. An expected result from the CGEGA would be a clear superiority of the Veo algorithm in the CGEGA hypoglycemic table. An additional benefit would be the better correspondence between the results and the CLSI guidelines,20 which recommend the use of trend accuracy analyses, including CGEGA.

Finally, an intriguing element of the paper is the mentioning of predictive alarms based on a Savitzky– Golay filter based on a moving polynomial fit, which theoretically should be more responsive to rapid time series changes and should result in shorter instrument delays than the commonly used moving average. The combination of this type of polynomial filtering and the improved hypoglycemia accuracy of the Veo algorithm would then result in more accurate hypoglycemia alarms. Indeed, it was estimated that predictive alerts with a 30 min predictive horizon would detect over 90% of hypoglycemic events.

In conclusion, the new Veo calibration algorithm appears superior to the standard PRT calibration, particularly in the critical hypoglycemic range. Clinically, this is an important achievement that has the potential to advance the quality of hypoglycemia alarms and, ultimately, of closed-loop control. This result, however, is based on a purely retrospective analysis and should be interpreted cautiously until confirmed by simulation studies testing the capabilities of the Veo algorithm in silico or by prospective data collection.

Abbreviations

BG
blood glucose
CGEGA
continuous glucose error grid analysis
CGM
continuous glucose monitoring
CLSI
Clinical and Laboratory Standards Institute
IG
interstitial glucose
MARD
mean absolute relative difference
PRT
Paradigm REAL-Time
SMBG
self-monitoring of blood glucose

References

1. Deiss D, Bolinder J, Riveline JP, Battelino T, Bosi E, Tubiana-Rufi N, Kerr D, Phillip M. Improved glycemic control in poorly controlled patients with type 1 diabetes using real-time continuous glucose monitoring. Diabetes Care. 2006;29(12):2730–2732. [PubMed]
2. Garg S, Zisser H, Schwartz S, Bailey T, Kaplan R, Ellis S, Jovanovic L. Improvement in glycemic excursions with a trans-cutaneous, real-time continuous glucose sensor: a randomized controlled trial. Diabetes Care. 2006;29(1):44–50. [PubMed]
3. Tamborlane WV, Beck RW, Bode BW, Buckingham B, Chase HP, Clemons R, Fiallo-Scharer R, Fox LA, Gilliam LK, Hirsch IB, Huang ES, Kollman C, Kowalski AJ, Laffel L, Lawrence JM, Lee J, Mauras N, O'Grady M, Ruedy KJ, Tansey M, Tsalikian E, Weinzimer S, Wilson DM, Wolpert H, Wysocki T, Xing D. The Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Continuous glucose monitoring and intensive treatment of type 1 diabetes. N Engl J Med. 2008;359(14):1464–1476. [PubMed]
4. Klonoff DC. Continuous glucose monitoring: roadmap for 21st century diabetes therapy. Diabetes Care. 2005;28(5):1231–1239. [PubMed]
5. Hirsch IB, Armstrong D, Bergenstal RM, Buckingham B, Childs BP, Clarke WL, Peters A, Wolpert H. Clinical application of emerging sensor technologies in diabetes management: consensus guidelines for continuous glucose monitoring (CGM) Diabetes Technol Ther. 2008;10(4):232–246. [PubMed]
6. Hovorka R. The future of continuous glucose monitoring: closed loop. Curr Diabetes Rev. 2008;4(3):269–279. [PubMed]
7. Rebrin K, Steil GM, van Antwerp WP, Mastrototaro JJ. Subcutaneous glucose predicts plasma glucose independent of insulimplications for continuous monitoring. Am J Physiol. 1999;277(3 Pt 1):E561–E571. [PubMed]
8. Steil GM, Rebrin K, Hariri F, Jinagonda S, Tadros S, Darwin C, Saad MF. Interstitial fluid glucose dynamics during insulin-induced hypoglycaemia. Diabetologia. 2005;48(9):1833–1840. [PubMed]
9. Boyne MS, Silver DM, Kaplan J, Saudek CD. Timing of changes in interstitial and venous blood glucose measured with a continuous subcutaneous glucose sensor. Diabetes. 2003;52(11):2790–2794. [PubMed]
10. Kulcu E, Tamada JA, Reach G, Potts RO, Lesho MJ. Physiological differences between interstitial glucose and blood glucose measured in human subjects. Diabetes Care. 2003;26(8):2405–2409. [PubMed]
11. Wentholt IM, Hart AA, Hoekstra JB, Devries JH. Relationship between interstitial and blood glucose in type 1 diabetes patients: delay and the push-pull phenomenon revisited. Diabetes Technol Ther. 2007;9(2):169–175. [PubMed]
12. Aussedat B, Dupire-Angel M, Gifford R, Klein JC, Wilson GS, Reach G. Interstitial glucose concentration and glycemia: implications for continuous subcutaneous glucose monitoring. Am J Physiol Endocrinol Metab. 2000;278(4):E716–E728. [PubMed]
13. Kovatchev B, Clarke W. Peculiarities of the continuous glucose monitoring data stream and their impact on developing closed-loop control technology. J Diabetes Sci Technol. 2008;2(1):158–163. [PMC free article] [PubMed]
14. Clarke WL, Anderson S, Farhy L, Breton M, Gonder-Frederick L, Cox D, Kovatchev B. Evaluating the clinical accuracy of two continuous glucose sensors using continuous glucose-error grid analysis. Diabetes Care. 2005;28(10):2412–2417. [PubMed]
15. Clarke WL, Kovatchev B. Continuous glucose sensors: continuing questions about clinical accuracy. J Diabetes Sci Technol. 2007;1(5):669–675. [PMC free article] [PubMed]
16. Kovatchev B, Anderson S, Heinemann L, Clarke W. Comparison of the numerical and clinical accuracy of four continuous glucose monitors. Diabetes Care. 2008;31(6):1160–2114. [PubMed]
17. Kovatchev BP, Gonder-Frederick LA, Cox DJ, Clarke WL. Evaluating the accuracy of continuous glucose-monitoring sensors: continuous glucose error-grid analysis illustrated by Therasense Freestyle Navigator data. Diabetes Care. 2004;27(8):1922–1928. [PubMed]
18. Diabetes Research in Children Network (DirecNet) Study Group. The accuracy of the Guardian RT continuous glucose monitor in children with type 1 diabetes. Diabetes Technol Ther. 2008;10(4):266–272. [PMC free article] [PubMed]
19. Keenan D, Cartaya R, Mastrototaro JJ. The accuracy of a new real-time continuous glucose monitoring algorithm. J Diabetes Sci Technol. 2009;4(1):111–118. [PMC free article] [PubMed]
20. Clinical and Laboratory Standards Institute. Clinical and Laboratory Standards Institute document POCT05-A. Wayne: Clinical and Laboratory Standards Institute; 2008. Performance metrics for continuous interstitial glucose monitoring; approved guideline.

Articles from Journal of Diabetes Science and Technology are provided here courtesy of Diabetes Technology Society