|Home | About | Journals | Submit | Contact Us | Français|
The objective of this study was to understand the level of performance of blood glucose monitors as assessed in the published literature.
Medline from January 2000 to October 2009 and reference lists of included articles were searched to identify eligible studies. Key information was abstracted from eligible studies: blood glucose meters tested, blood sample, meter operators, setting, sample of people (number, diabetes type, age, sex, and race), duration of diabetes, years using a glucose meter, insulin use, recommendations followed, performance evaluation measures, and specific factors affecting the accuracy evaluation of blood glucose monitors.
Thirty-one articles were included in this review. Articles were categorized as review articles of blood glucose accuracy (6 articles), original studies that reported the performance of blood glucose meters in laboratory settings (14 articles) or clinical settings (9 articles), and simulation studies (2 articles). A variety of performance evaluation measures were used in the studies. The authors did not identify any studies that demonstrated a difference in clinical outcomes. Examples of analytical tools used in the description of accuracy (e.g., correlation coefficient, linear regression equations, and International Organization for Standardization standards) and how these traditional measures can complicate the achievement of target blood glucose levels for the patient were presented. The benefits of using error grid analysis to quantify the clinical accuracy of patient-determined blood glucose values were discussed.
When examining blood glucose monitor performance in the real world, it is important to consider if an improvement in analytical accuracy would lead to improved clinical outcomes for patients. There are several examples of how analytical tools used in the description of self-monitoring of blood glucose accuracy could be irrelevant to treatment decisions.
When performed and utilized properly, monitoring of blood glucose (BG) permits people with diabetes to determine their BG level and use the information as part of their treatment program. The overall performance of self-monitoring of blood glucose (SMBG) systems is a combination of the analytical performance of the instrument, quality of the test strips, and proficiency of the user. Considerations for improving the accuracy of glucose monitoring systems emphasize technical improvements and improved patient education to decrease user error.1 The term “accuracy” as applied to analytical performance is defined by the International Organization for Standardization (ISO) as “the difference between the expectation of measurement results and the true value of the measured quantity.” In essence, it is a measure of the difference between obtained results (by the blood glucose monitor) and the true value (determined by an accepted reference method). To our knowledge, an internationally accepted reference method for the determination of whole BG concentration does not exist. There is currently a lack of consensus on the performance standards for blood glucose monitors and whether target values of an expanded ISO 15197 standard would be appropriate.
The authors undertook an analysis of the literature to understand the performance of blood glucose monitors in the real world. This article reports on the review of the published literature to provide information on current blood glucose monitor performance. Also discussed are differences between what level of accuracy is needed for appropriate analytical performance and appropriate clinical performance and why different settings might require different levels of accuracy.
Medline (January 2000–October 2009) was searched for eligible articles using combinations of the following search terms: (1) diabetes mellitus, or type 1 diabetes mellitus, or type 2 diabetes mellitus; (2) blood chemical analysis, or blood glucose, or blood glucose self-monitoring, or point-of-care systems; (3) calibration, or laboratory techniques and procedures, or quality control, or reference standards, or reference values, or reproducibility of results; and (4) humans. The reference lists of included studies were also searched.
Titles and abstracts of identified citations were screened, and articles were identified as eligible based on the following criteria. Inclusion criteria were review articles of BG accuracy, original articles that reported the performance of blood glucose meters in laboratory settings, original articles that reported the performance of blood glucose meters in clinical settings, and simulation modeling of BG. The following information was collected from eligible articles for each of the categories: (1) review articles of blood glucose accuracy—specific factors affecting the accuracy evaluation of blood glucose monitors; (2) original articles that reported the performance of blood glucose meters in laboratory settings—blood glucose meters tested, blood sample, meter operators, and performance evaluation measures; (3) original articles that reported the performance of blood glucose meters in clinical settings—blood glucose meters tested, setting, sample (number, diabetes type, age, sex, and race), duration of diabetes, years using a glucose meter, insulin use, recommendations followed, and performance evaluation measures; and (4) simulation modeling of blood glucose—sample, results, and conclusions.
Comprehensive literature searches identified 563 articles. Articles were screened and 31 articles met eligibility criteria. Twenty-three of the 31 articles (74%) were published in the past 4 years.
Five of the six review articles focused on factors affecting evaluation of the accuracy of blood glucose monitors.1–5Factors can be categorized as patient/user knowledge, methods and analyses, sources of interference, and reporting of results. Patient/user knowledge factors include educating patients in the proper techniques of glucose meter use,2,3 proper handling and storage of test strips,2,5 proper storage of control solution,5 meter cleanliness,5timing of sample collection after eating or insulin dose,2–5 size and placement of blood sample,5 and removal of blood from the strip.5 Education would provide an opportunity to reduce operator error2,4 and falsification of results by the patient.3 A poor user interface4 can also impact proper glucose meter use. Factors related to the methods and analyses include blood versus plasma/serum glucose,1,2,4 source of sample (arterial, capillary, and venous),1,2,4 calibration of instrument,1,5 different enzymatic methods,2 between-lot variability of glucose strips,1,4,5 analytic imprecision of laboratory reference method,2 and study design.2,6
Several substances and patient factors are sources of interference. Substances include galactose,3 xylose,3 sodium fluoride,1 glutathione,1 cysteine,1 uric acid,1,3 hemolysis,1,2,5 and drugs (e.g., ascorbic acid,1–5 aspirin,2,5 acetaminophen,1,3–5 dopamine,4 icodextrin,4,5 maltose,3,5 mannitol,4 and tolbutamide1). Patient factors include hypotension,1,4 hypertriglyceridemia,1,2 hypothermia,4 pH,4 oxygen (e.g., hypoxia and oxygen therapy),1–4 hematocrit (e.g., anemia and polycythemia),1,3–5 and bilirubin.3 Factors of reporting results of accuracy include statistical methods (e.g., correlation coefficients, linear regression, percent deviation, and mean differences),1,2 bias plots,2 and error grids.1,2,4 A sixth review article examined best practices for conducting and reporting glucose monitor evaluation studies.6 A checklist is presented in the article that outlines an approach for a protocol that is evidence based and provides enough details for reproducibility of methods and results.6
Fourteen articles reported on the performance of blood glucose meters in laboratory settings7–20 (Table 1). In each article, 17 to 3019 blood glucose meters were tested. Most commonly, 3,11,17,18 4,7,9,14,16,20 or 510,12,13 blood glucose meters were compared. Meter operators were specified in only three of the articles.8,9,13 The most common performance evaluation types included accuracy (e.g., bias,9,10,12,15,18,19 Bland–Altman plots,7,8,10,14–16,20 or error grid analysis8–10,15,18), analytical range,9,12,16 effect of chemical interference on meter accuracy,7,8,12,20 effect of hematocrit interference on meter accuracy,7,11,12,14,17,19,20 effect of PO2 on meter accuracy,14,17 locally smoothed median absolute difference curves,8,14,15 precision,7,8–12,16,20 and performance criteria (e.g., ISO 15197,8,11,14–16 The Netherlands Organization for Applied Scientific Research,19 or other professional and regulatory agencies9). Performance evaluation measures of blood glucose meters in laboratory settings are presented in Table 2. Two9,18 of the five8–10,15,18 studies that provided results from an error grid analysis (EGA) included measurements that fell outside of zones A and B.
Nine articles reported 12 studies on the performance of blood glucose meters in clinical settings21–29 (Table 3). In each protocol, 123,26,29 to 2121 blood glucose meters were tested. Settings included hospital,22,26,29 outpatient clinics,21,23–26 and home.27,28 A total of 1990 patients participated in the studies (median 101, range 32 to 715). The patient used a blood glucose meter in all but three studies, where a health care provider used a meter instead.22,23,29 Each of the studies involved people with type 1 or type 2 diabetes except for studies of women with gestational diabetes,24 of children with only type 1 diabetes,22 and of lay users and health care professionals.26,29 Five studies included children22,23,25,27,28; however, only 1 of the studies involved children exclusively.22 Two of the studies applied their findings to American Diabetes Association guidelines21,24 and almost all other studies applied findings to the ISO 15197 standard.22,23,25,26,28,29 Performance measures of blood glucose meters in clinical settings are presented in Table 4. Only 5 of the studies (presented in three of the articles)23,25,26 provided results from an EGA. All measurements in these studies fell into zones A and B.
Two articles reported the use of simulation to investigate insulin dosing errors.30,31 One study calculated that insulin dosage errors would occur 8 to 23% of the time in a meter with a total analytical error of 5%.30 Insulin dosage errors also were calculated to occur 16 to 45% of the time in a meter with a total analytical error of 10%.30 This study concluded that glucose meters meeting current quality standards permit a large percentage of administered doses to differ from intended doses.30 The other study focused on intravenous insulin therapy in critically ill patients and used a sample of 29,920 glucose values to simulate glucose values with error.31 In the simulation, when 10, 15, or 20% total error was modeled, one category insulin errors occurred with 39, 46, or 46% of simulated glucose values and two category insulin errors occurred with 3, 9, or 16% of simulated glucose values.31 The study concluded that permitting a 10% total error in glucose measurement would result in safer management of patients on intravenous insulin therapy.31
Self-blood glucose monitors (SBGM) were introduced widely during the early 1980s and became commonly used in the 1990s as a replacement for urine testing as a means for patients with diabetes to determine their current level of glycemia. Patients were taught to use these SBGM readings to guide their decisions regarding immediate treatment. While analytical or statistical accuracy of SBGM systems is necessary for Food and Drug Administration (FDA) approval and may be useful for demonstrating relative differences among systems, presentations of accuracy using analytical terms alone may not describe accuracy in its entirety.
Several examples of how analytical tools used in the description of SBGM accuracy could be irrelevant to treatment decisions are presented. First, correlation coefficients, which describe relationships between two variables—reference BG and SBGM BG— can be close to unity and highly significant when large numbers of data points are analyzed, yet individual data point may differ by large amounts.32 Pohl and colleagues33 have shown that even when the correlation coefficient of a large set of reference and SBGM determinations is highly significant across the entire BG range, correlation coefficients for that same data may differ significantly in the three critical BG ranges—hypoglycemia, euglycemia, and hyperglycemia. Linear regression equations present the slope of the “best-fitting” line relating two sets of data, but even when that slope approaches unity, the relationship between two data points is not assured. ISO standards acknowledge that percentage differences between reference and SBGM BG values do not apply across the entire BG reference range; however, these criteria are not sufficient to prevent the possibility of serious errors in treatment decision making.34 For instance, if the reference value is 74 mg/dl, a level that would signal impending hypoglycemia, a SBGM BG value would be considered accurate if it were between 59 and 89 mg/dl. These two values signal entirely different clinical responses. Likewise if the reference BG were 76 mg/dl, then a 20% deviation of SBGM values from 51 to 91 mg/dl would be considered accurate.
These examples clearly suggest that traditional analytical methods, even though widely accepted as standards for reporting accuracy of SBGM systems, may confer a level of accuracy that could complicate a patient's ability to achieve BG targets, including the detection and prevention of hypoglycemia.
Patients and clinicians use SBGM systems to guide very specific clinical decisions. Thus a different method for describing the accuracy of SBGM is needed to reduce potential clinical errors that could be associated with traditional statistical analyses. EGA was the first method developed to quantify the clinical accuracy of patient-determined BG values.32 EGA categorizes the relationship between a patient-generated BG level and a reference BG levels in terms of the clinical status that would result from a treatment decision based on the patient-generated result. Subsequently, Parkes and colleagues35 developed the consensus error grid (CEG), a similar method for describing the accuracy of SBGM based on clinical decision making. Both of these analyses are designed to emphasize the importance of obtaining clinically accurate information across the entire BG range (hypoglycemia, euglycemia, and hyperglycemia).36
The EGA is a graph of reference vs SBGM BG data pairs, which is divided into five zones of clinical accuracy (Figure 1). The basic assumptions of EGA are that the target BG range is between 70 and 180 mg/dl and that any patient-generated BG value outside that range will be treated according to protocols or algorithms selected by the health care provider and the patient. Zone A (upper and lower) data pairs represent patient-generated values, which are within 20% of the reference values and/or ≤70 mg/dl when the reference is ≤70 mg/dl. Zone A points are categorized as clinically accurate because they could lead to accurate treatment decisions. Zone C (upper and lower) data pairs represent possible “overcorrection errors,” as patient-generated values in these zones could trigger treatment responses that might result in the subsequent BG value being outside the target range. Zone D (upper and lower) values are “failure to treat” errors because the patient-generated values are within the target range when the reference value is either low (≤70 mg/dl) or high (≥240 mg/dl). Zone E values are “erroneous errors” where the patient-generated values are either high (>180 mg/dl) when the reference is low (≤70 mg/dl) or low (≤70 mg/dl) when the reference is high (>240 mg/dl). Patient self-treatment based on these errors could result in serious hypoglycemia or hyperglycemia. Zone B data pairs are those where the patient-generated value deviates from the reference by more than 20% but may not result in clinically significant treatment errors. They are designated clinically acceptable.
Since the late 1980s, EGA has been used by most manufacturers of SBGM devices to demonstrate the clinical accuracy of their systems and reported along with more traditional statistical analyses to the FDA as part of premarketing applications.37 In the original presentation of the EGA, results from a variety of SBGM were presented.32 In no case was the clinically accurate/acceptable (zones A+B) percentage less than 94%. Indeed, even when results of visually interpreted BG strips were plotted on the EGA, few errors in clinical decision making would have been expected. Thus, it appears, from over 20 years of data collection, that the clinical accuracy of SBGM systems analyzed using either EGA or CEG is sufficient to permit patients to make appropriate treatment. SBGM systems were used by subjects participating in the Diabetes Control and Complications Trial (DCCT) to achieve intensive control of their BG levels and to reduce their risk of diabetic complications.38
It is important to point out that clinical accuracy depends on the BG target range. Although the target range used most commonly for FDA submissions of clinical accuracy of SBGM systems using EGA is 70 to 180 mg/dl, that range is not fixed. It may differ depending on the clinical situation and the treatment goals of an individual patient. In the original description of EGA, an example is presented where the target range of a data set is changed from 70–180 to 60–120 mg/dl, a target being used as a goal for managing glycemia during pregnancy. As a result of that change, the clinically accurate/acceptable (zones A+B) percentage increased while the clinically inaccurate (zones C, D, and E) percentage decreased.32 Thus selecting a different target range for BG alters the size and/or position of the EGA zones and the clinical accuracy of a given SBGM system, but does not change its statistical accuracy. A careful examination of the EGA (Figure 1) and its zones of clinical accuracy permits one to evaluate the potential consequences of selecting different target ranges. For instance, use of a narrow target range (80–110 mg/dl) for hospitalized intensive care patients would shift lower zone C (overcorrection) as well as lower zone E (erroneous) to the left such that the reported fivefold increase in hypoglycemia might have been anticipated.39
Because the clinical accuracy of SBGM assumes that the patient will take treatment action to return his/her BG into the target range, it is clear that patient education and technical performance play a role in the clinical accuracy of SBGM systems. These patient factors and their contribution to accuracy have been reviewed in this article.
The American Diabetes Association has suggested that SBGM systems be developed to achieve an analytical plus user error of less than 10% at BG levels between 30 and 400 mg/dl.40 The analytical error goal for such SBGM systems is 5% or less. We have reviewed recent publications describing the analytical and clinical performance accuracy of current systems and the numerous factors, physiologic, environmental, and educational, that affect accuracy. Researchers and clinicians must ask the following obvious questions. “What are the potential benefits to patients that would accompany an improvement in analytical accuracy? Would an improvement in analytical accuracy be logically accompanied by any improvement in clinical outcomes, such as reduced morbidity and mortality in hospitalized patients, or lower hemoglobin A1c (HbA1c) levels, reductions in glycemic verticality, and/or reductions in acute and chronic complications in outpatients?”
Simulation studies suggest that a reduced SBGM analytical error could be associated with fewer incorrect insulin doses given to ICU patients being treated with tight glucose control protocols.39 However, in these simulations, SBGM error had to be reduced to <2% to ensure 95% correct insulin dosing. A lower analytical error might also permit a more rapid and accurate diagnosis of diabetes based on a single fasting BG level.
As stated earlier, SBGM systems were not developed to be substitutes for the precise analytical instruments used to determine BG in hospital laboratories. They were developed to provide immediate BG information to patients with diabetes so that they might make their own treatment decisions. SBGM systems have been used since the late 1970s with varying degrees of success by educated and motivated patients in clinical trials such as the DCCT and in routine self-management. The authors are unaware of any clinical studies that demonstrate a difference in clinical outcomes—HbA1c, glycemic variability, pregnancy, hypoglycemia, diabetic ketoacidosis, or chronic complications—when subjects used SBGM systems with different analytical accuracy. It is possible that this may be due to the limited amount of information provided by a single BG determination. Without information regarding recent BG values and the current rate and direction of BG change, as provided by continuous glucose monitoring technology, reductions in the analytical error of single BG determinations may be insufficient to affect changes in clinical outcomes.