PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jdstLink to Publisher's site
 
J Diabetes Sci Technol. 2016 July; 10(4): 939–946.
Published online 2016 March 16. doi:  10.1177/1932296816638854
PMCID: PMC4928233

Performance of a Blood Glucose Monitoring System in a Point-of-Care Setting

Abstract

This study assesses and demonstrates that CONTOUR® XT-BGMS (CXT-BGMS) complies with the requirements of the German (RiliBÄK) and Swiss (QUALAB) quality control guidelines for point-of-care testing (POCT) and fulfills the ISO15197:2013 accuracy limits criteria under the routine conditions of a hospital point-of care setting. This single-center study was conducted in Switzerland using 105 venous blood samples from hospitalized patients. Each sample was tested in comparison to the hexokinase reference method. Compliance with POCT guidelines was assessed by daily BGMS measurements using control solutions. Accuracy of CXT-BGMS according to ISO limits was 98.41%. All control measurements were within the limits defined by RiliBÄK (within ± 11% of target values and root mean square error [RMSE] within RMSE limits), and QUALAB (within ± 10% of target values).

Keywords: BGMS, Contour XT, POCT, QUALAB, RiliBÄK, ISO 15197

In Europe, BGMS are concurrently approved and used both for glucose self-monitoring by lay users, as well as for blood glucose point-of-care testing (POCT) by health care professionals.1,2 Handheld BGMS are easy to operate, and they deliver results quickly. This is of particular relevance in a hospital setting, where medically urgent cases may require fast and efficient treatment.3

To ensure patient safety, specific quality control guidelines developed in Europe define maximum permissible deviations from the target values for control measurements in point-of-care settings:

In Germany, internal quality control guidelines of the Bundesärztekammer (Guidelines of the German Chamber of Physicians, RiliBÄK) stipulate that results should be within ± 11% of the target value for the control. In addition the root mean square errors (RMSEs) of glucose measurements must be less than or equal to defined limits based on sample size and target value. Control measurements must be performed at least once a week with 2 control solutions of different glucose levels, and when measurements are not within the permissible range, corrective actions must be taken as indicated by the manufacturer.4

In Switzerland, the internal quality guidelines of the Schweizerische Kommission für Qualitätssicherung im Medizinischen Labor (Swiss Commission for Quality Assurance in Medical Laboratories, QUALAB) stipulate for blood glucose measurements in a POCT setting that results must be within ± 10% of the control solution target values. Quality controls must be performed periodically, at least every 2 weeks or more frequently, if so indicated by the manufacturer.5

Apart from internal quality controls, BGMS must fulfill International Organization for Standardization (ISO) norms. Currently, ISO 15197 guideline provides the European regulatory standard accuracy assessment of BGMS.6

The objective of this postmarketing surveillance was to provide evidence of compliance with German and Swiss POCT quality control requirements and ISO 15197:2013 accuracy limits fulfillment for the CONTOUR® XT BGMS (CXT-BGMS), when tested under the routine conditions of a point-of-care setting.

Methods

Study Design

This study was conducted in the Cantonal Hospital Aarau, Switzerland, under local routine conditions. As exclusively deidentified leftover blood samples were used in this study, no informed consent or ethical approval process was required according to local laws.

105 leftover venous blood samples from hospitalized adult patients who were prescribed blood tests (including at least a glucose test and hematocrit analysis) were used. Patients had their blood taken by venipuncture per routine practice. Samples were exclusively collected in tubes containing lithium heparin. Samples with hematocrit results outside the CXT-BGMS range (>70%)7 were considered nonevaluable.

Blood glucose levels were measured using the CXT-BGMS, and the local, available hexokinase method (Dimension® Vista 1500, Siemens AG, Erlangen. Germany). Measurements were exclusively carried out by trained staff without involving any patient participation. Per local routine practice all measurements were performed immediately (BGMS) or on the same day (laboratory test) the sample had been obtained. One drop was dispensed onto a glass slide immediately before each test with the BGMS. As this study was largely conducted under local routine conditions, it did not attempt to fulfill formal methodological ISO requirements (ISO 15197:2013, section 6.3).6

To test performance across the entire measuring range of the BGMS (0.6-33.3 mmol/L), 10% of samples were required to have glucose concentration levels <4.440 mmol/L, and 10% had to be in the glycemic range >13.877 mmol/L. As the incoming samples of regular patients did not provide these extreme glucose ranges, blood samples were modified either by glycolysis or by glucose supplementation.

Blood Glucose Monitoring System

The CXT-BGMS (Ascensia Diabetes Care, Basel, Switzerland), which uses Contour® Next (Ascensia Diabetes Care, Basel, Switzerland) test strips, is intended both for self-testing by diabetic patients and for use by health care professionals, and may be used with fresh capillary blood and venous whole blood (Contour XT User guide Rev 01/2013). Within this study, the BGMS was solely used with venous blood samples. For each CXT-BGMS device a defined strip lot was used, Lot A, B or C. Each BGMS measurement was compared with the corresponding hexokinase method measurement of the same sample. To avoid glycolysis, samples were centrifuged within 10 minutes after BGMS measurements, and hexokinase measurements were performed with the plasma samples.

In accordance with—and exceeding—the requirements of QUALAB and RiliBÄK,4,5 control testing on the CXT-BGMS was conducted each day of the study prior to any sample testing. One control measurement per strip lot and per control solution were performed with each of the 3 BMGS used.

Local Hexokinase Laboratory Method

The reference instrument used was the Dimension Vista 1500 (Siemens Dimension Vista®, Siemens AG, Erlangen, Germany), which utilizes the hexokinase method for glucose measurement. The hexokinase method is traceable to the BGMS calibration method stated by the manufacturer (Yellow Springs Analyzer, Inc, Yellow Springs, OH, USA). This reference method is common for quantitative assessment of blood glucose levels within Europe. Moreover, it is widely used by technical certification providers for evaluation of BGMS (such as TNO/TÜV Rheinland AG [www.tuv.com], Germany, or SKUP [www.skup.nu]).

Prior to the study and during each day of the study, measurements of 4 glucose serum control levels (traceable to National Institute of Standards and Technology [NIST] standards) were performed with the reference analyzer and results compared to predetermined limits.

Assessment and Analysis

Basic descriptive statistics including absolute and relative frequency distributions and 2-sided 95%-confidence intervals were used for description of the accuracy results.

For RiliBÄK the RMSE was computed as follows:

RMSEk=1ni=1n(xiτk)2

k level; τk target values; xiMeasurements; n Number used to calculate the individual results

Regression analysis

Regression analysis utilizing weighted least squares (WLS), with weight (w) function8

w=1(Lab)2

used to fit a straight line to data pairs—BGMS and reference method (Lab). Regression statistics were computed with outliers (data not shown).

Error Grid analysis

CEG analyses9,10 were performed for all samples using a Microsoft Excel™ VBA macro program (Microsoft Corporation, Redmond, WA, USA).

Results

Sample characteristics

The glucose range of the 105 evaluable samples was 2.2 to 28.4 mmol/L, and hematocrit ranged from 21% to 55%. 63 samples had glucose concentrations ≥ 5.551 mmol/L, 12 of which were modified samples with concentrations >13.877 mmol/L. 42 samples had glucose concentrations <5.551 mmol/L, 12 of which were modified samples with concentrations <4.440 mmol/L.

Trueness and Precision of the Local Laboratory Hexokinase Reference Method

All results obtained with the 4 glucose serum controls levels prior to the study and during the study were within the limits established by the serum manufacturer (Ascensia Diabetes Care USA, Parsippany,, USA), and by RiliBÄK (data not shown).

POCT Quality Controls of CXT-BGMS

A total of 26 control tests were obtained per control solution level (“low” and “normal”) and per strip lot using the 3 CXT-BGMS. All results were within the tolerance limits specified on the strip labels by the manufacturer. The median of the tolerance limits were considered as the target values.

Analyzing the relative differences in% (RD) from the targets, all results satisfied RiliBÄK (errors within ± 11% from the target values) (Figure 1).

Figure 1.
Shewhart control charts11 showing the individual relative difference (RD) (%) from the target of the low (left) and normal (right) control solution levels. Upper and lower horizontal lines represent the accuracy range defined by the RiliBÄK guideline ...

Additional analysis of RMSE showed that all results were below the critical values (maximum allowable values for the RMSEs per target (RMSEcrit) adjusted for sample size and assuming controls results are normally distributed). Table 1 shows RMSE and RMSEcrit expressed as a percentage of the target values.

Table 1.
RMSE of CXT-BGMS Results and Critical RMSE Values According to RiliBÄK.

In addition, the control results were displayed according to QUALAB (Figure 2). All meter results were within ± 10% of the control solutions targets, calculated as the mean of the meter results for each lot and level. In addition, control and warning limits for series of results (2 and 3 standard deviations respectively from the mean value of series) specified by QUALAB were applied to the results.

Figure 2.
Shewhart control charts showing meter results according to QUALAB obtained for low (left) and normal (right) control levels with each strip lot, A, B, C. The targets are shown as the mean of the results (gray line). The warning (±2× standard ...

Accuracy of the BGMS

Out of 315 CXT-BGMS measurements, 310 (98.41%) satisfied ISO 15197:2013 accuracy limits (± 0.833 mmol/L for samples with laboratory reference concentrations <5.551 mmol/L or ± 15% with samples ≥ 5.551 mmol/L) of the hexokinase method results (Table 2). No error codes occurred with the CXT-BGMS during the study.

Table 2.
Accuracy Results for the BGMS per Strip Lots, and Overall.

The corresponding error distributions are displayed in Table 3.

Table 3.
Error Distributions for Glucose Measurements (ISO 15197:2013 Requirements).

A post hoc analysis was performed to evaluate the accuracy of specific glucose ranges by representation of results in modified Bland-Altman plots and calculating MARD (see the appendix).12,13 A polar representation of the modified Bland-Altman plots (Radar plot) was also performed.12 In this representation, the closer a point to the center, the more accurate it is (Figure 3).

Figure 3.
Radar plot. The 2 diagonal dashed lines mark the area of hexokinase measurements <5.551 mmol/L; data points inside this area are expressed as absolute differences from the reference value in mmol/L. For blood glucose concentrations ≥ 5.551 ...

Error Grid Analysis

CEG analysis9 showed that 99.4% (313 out of 315) of the BGMS results were within Zone A (“no effect on clinical action”). Only 0.6% (2 out of 315) were within Zone B (“altered clinical action with little or no effect on clinical outcome”) (Figure 4).9

Figure 4.
Consensus error grid (CEG) analysis assigning the error of a BGMS measurement to 1 of 5 increasing clinical risk zones. The y-axis depicts the result of measurement with the BGMS; the x-axis shows the results of the reference hexokinase method measurement. ...

Discussion

The present study demonstrated that the performance of the CXT-BGMS meets the requirements of the German (RiliBÄK) and Swiss (QUALAB) quality control guidelines for POCT under the hospital routine conditions of a point-of-care setting. Notably, a nonroutine aspect was that quality testing was performed daily, which is more frequently than stipulated by both RiliBÄK and QUALAB. Thus, more stringent controls were applied in this study. The daily control measurements within this study may allow for a rough simulation on the BGMS performance over a longer period of time, by extrapolating from the daily controls to the weekly control measurements stipulated by the guidelines. Moreover, this study demonstrated that the BMGS fulfilled the accuracy limits established by ISO 15197:2013 and the novel CEG criterion. None of the reading accuracy errors were of clinically significant relevance. Of note, the current study was not designed according to all methodological requirements described in ISO 15197:2913 section 6.3,6 but it followed largely routine procedures at the local hospital. Thus, venous blood samples were used and single glucose measurements performed with each of the 3 BGMS. Another point to acknowledge is that the number of samples was not distributed at the defined percentages stated in the ISO standard for the different glucose concentration ranges. Instead, and although not being part of the local hospital routine, 20% of the samples were required to fall in defined high and low glucose ranges and were obtained by sample modification.

As extreme values may coincide with severe clinical symptoms and complications,14,15 studies usually use modified samples to not compromise patient safety.16,17 Moreover, results in the low glycemic range tend for some BGMS to worsen and thus warrant particular scrutiny.16 In this study, at the defined middle and highglucose ranges (5.55-11.10 mmol/L, >11.10 mmol/L) 100% of all measurements were within ISO accuracy limits; whereas in the low glycemic range (<5.55 mmol/L) 96.30% of measurements were within these limits. CEG results mirrored this observation: All but 2 measurements (99.4%) were in Zone A (“no effect on clinical action)—and these 2 measurements (Zone B, “altered clinical action with little or no effect on clinical outcome”) were in the low glycemic range. Thus, albeit accuracy was somewhat lower, ISO accuracy limits were also fulfilled in this range.

The data of this study support several published reports on the accuracy and reliability of the CXT-BGMS. It has been demonstrated that this BGMS fulfills the standards as laid out in ISO 15197 section 6.3, including the more stringent requirements introduced in 2013.6,7,18 Noteworthy comparative studies reported good performance of CXT-BGMS when ISO 15197:2013 accuracy limits were applied.16,17,19

In a recent multicenter study conducted in 21 hospitals under routine conditions using 2100 leftover blood samples,20 CXT-BGMS results were remarkably congruent with the present data. The ISO accuracy limits and CEG results similarities corroborate the generalizability of the performance data of the CXT-BGMS in different hospital environments.

Within Europe, the concept of POCT in the field of glucose measurement is generally embraced. Given the undeniable advantages of obtaining blood glucose results quickly, more frequent BGMS use by health care professionals would be desirable, either directly at the patients’ bedside, or in a point-of-care settings as the one in this study. In any case, fulfillment of stringent quality guidelines must always remain paramount in measuring blood glucose levels in POCT.4,5

Conclusions

This study demonstrates that the CXT-BMGS meets the local applicable quality control requirements for POCT and fulfills ISO 15197:2013 accuracy limits when tested under the routine conditions of a point-of care setting.

Acknowledgments

The authors would like to thank Christoph Engler, MD and GKM Gesellschaft für Therapieforschung mbH for support in completion of this manuscript.

Appendix

Accuracy Within Specific Glucose Ranges

Three categories (low, middle, high) of glucose levels were defined (<5.55 mmol/L, 5.55-11.10 mmol/L, >11.10 mmol/L). Modified Bland-Altman plots (Figure A1) show, that all measurements (100%) in the “middle” and “high” glucose categories were within the ISO accuracy limits. In the “low” glucose category (<5.55 mmol/L) 96.30% of samples were within ISO accuracy limits.

Figure A1.

An external file that holds a picture, illustration, etc.
Object name is 10.1177_1932296816638854-fig5.jpg

Bland-Altman plots for specific glucose ranges (upper left, <5.55 mmol/L, lower left, 5.55-11.110 mmol/L, upper right, >11.10 mmol/L, lower right overall samples). The y-axis depicts the difference (D) between BGMS result (Meter) and the hexokinase reference result (Lab), and the x-axis is the blood glucose level as assessed by the reference hexokinase method (Lab) (mmol/L). The lower and upper accuracy limits (LL, UL, expressed in mmol/L) depicted as green and blue lines in the plots are according to ISO 15197:2013 either ± 0.833 mmol/L (hexokinase < 5.551 mmol/L) or ± 15% of hexokinase result (hexokinase ≥ 5.551 mmol/L).

Table A1.

ParameterUnitGlucose range (mmol/L)nMeanSDLCLUCL
MADmmol/L<5.551350.220.2440.180.26
MARD%≥5.551804.092.5753.714.46
MARD%5.55 ≤ x ≤11.11324.162.5543.724.60
MARD%>11.1483.872.6503.104.64

MARD Statistics.

LCL, lower confidence limit; MAD, mean absolute difference; MARD, mean absolute relative difference; UCL, upper confidence limit.

An additional post hoc analysis was performed to assess BGMS accuracy of different glucose ranges by calculating the mean absolute difference (MAD) and mean absolute relative difference (MARD) between the BGMS and the reference hexokinase method. MARD is the mean of the Absolute value of Relative (%) Difference (ARD) between a BGMS result (BG) and the corresponding hexokinase (H) result.

ARD=100x|BGH|H

MAD is the mean of the absolute value (mmol/L) of the difference (AD) between a BGMS result (BG) and the corresponding hexokinase (H) result and it is a more accurate calculation for values <5.55 mmol/L.

AD = |BG − H|

Lower MAD and MARD values indicate smaller differences between reference values and meter values.

MARD analysis was computed for the 3 glucose categories (low, middle, high). Mean absolute difference (MAD) was calculated for the low glucose range <5.55 mmol/L. Table A1 shows the results, along with the standard deviations (SD) of ARD, and 95% confidence interval limits (LCL and UCL) for MARD. MARD was smallest for the high glucose category >11.1 mmol/L.

Footnotes

Abbreviations: BG, blood glucose; BGMS, blood glucose monitoring system; CEG, consensus error grid; ISO, International Organization for Standardization; LCL, lower confidence limit; MARD, mean absolute relative difference; MAD mean absolute difference; POCT, point-of-care testing; QUALAB, Richtlinie der Schweizerischen Kommission für Qualitätssicherung im medizinischen Labor; RiliBÄK, Richtlinien der Bundesärztekammer zur Qualitätssicherung laboratoriumsmedizinischer Untersuchungen; RMSE, root mean square error; UCL, upper confidence limit.

Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: BFT, SP, and TP are full-time employees of Ascensia Diabetes Care.

Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this study and necessary equipment to conduct the study (devices, labeling information, case report forms, methodology and logistics) was provided by Bayer Switzerland AG, the predecessor-in-interest of Ascensia Diabetes Care, Zurich, Switzerland.

References

1. Tonyushkina K, Nichols JH. Glucose meters: a review of technical challenges to obtaining accurate results. J Diabetes Sci Technol. 2009;3(4):971-980. [PMC free article] [PubMed]
2. Klonoff DC. Point-of-care blood glucose meter accuracy in the hospital setting. Diabetes Spectrum. 2014;27(3):174-179. [PMC free article] [PubMed]
3. Junker R, Schlebusch H, Luppa PB. Point-of-care testing in hospitals and primary care. Deutsches Ärzteblatt International. 2010;107(33):561-567. [PubMed]
4. Bundesärztekammer. Richtlinie der Bundesärztekammer zur Qualitätssicherung laboratoriumsmedizinischer Untersuchungen. 2013.
5. Schweizerische Kommission für Qualitätssicherung immedizinischen Labor, Richtlinie zur internen Qualitätskontrolle; Anhang zum Konzept für Qualitätssicherung im medizinischen Labor (Konzept QUALAB), version2.8. 2014.
6. ISO 15197. In vitro diagnostic test systems—requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. 2013.
7. Bernstein R, Parkes JL, Goldy A, et al. A new test strip technology platform for self-monitoring of blood glucose. J Diabetes Sci Technol. 2013;7(5):1386-1399. [PMC free article] [PubMed]
8. Draper NR. Applied Regression Analysis. 3rd ed. New York, NY: John Wiley; 1998.
9. Parkes JL, Slatin SL, Pardo S, Ginsberg BH. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care. 2000;23(8):1143-1148. [PubMed]
10. Krouwer JS, Cembrowski GS. A review of standards and statistics used to describe blood glucose monitor performance. J Diabetes Sci Technol. 2010;4(1):75-83. [PMC free article] [PubMed]
11. Grant EL, Leavenworth R. Statistical Process Control. New York, NY: McGraw-Hill; 1980.
12. Simmons DA. How should blood glucose meter system analytical performance be assessed? J Diabetes Sci Technol. 2015;10:178-184. [PMC free article] [PubMed]
13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307-310. [PubMed]
14. MacLeod KM, Hepbum DA, Deary IJ, et al. Regional cerebral blood flow in IDDM patients: effects of diabetes and of recurrent severe hypoglycaemia. Diabetologia. 1994;37(3):257-263. [PubMed]
15. Wright RJ, Newby DA, Stirling D, et al. Effects of acute insulin-induced hypoglycemia on indices of inflammation: putative mechanism for aggravating vascular disease in diabetes. Diabetes Care. 2010;33(7):1591-1597. [PMC free article] [PubMed]
16. Freckmann G, Pleus S, Link M, et al. Accuracy evaluation of four blood glucose monitoring systems in unaltered blood samples in the low glycemic range and blood samples in the concentration range defined by ISO 15197. Diabetes Technol Ther. 2015;17:625-634. [PubMed]
17. Freckmann G, Baumstark A, Schmid C, et al. Evaluation of 12 blood glucose monitoring systems for self-testing: system accuracy and measurement reproducibility. Diabetes Technol Ther. 2014;16(2):113-122. [PubMed]
18. Pflug B, Warchal-Windham ME, Goldy A, Chu A. Analytical accuracy evaluation of the CONTOUR® XT blood glucose meter. J Diabetes Sci Technol. 2013;7(1):281-284.
19. Lemke C, Petruschke T, Wallace JF, et al. Comparative accuracy evaluation of the CONTOUR XT® blood glucose monitoring system. In: 47th Annual Diabetes Congress Stuttgart, Germany: Deutsche Diabetes Gesellschaft; 2012:16-19.
20. Bedini JL, Wallace JF, Petruschke T, Pardo S. A multicenter performance evaluation of a blood glucose monitoring system in 21 leading hospitals in Spain. J Diabetes Sci Technol. 2015;10:93-100. [PMC free article] [PubMed]

Articles from Journal of Diabetes Science and Technology are provided here courtesy of Diabetes Technology Society