|Home | About | Journals | Submit | Contact Us | Français|
Point-of-care instruments for the measurement of hemoglobin A1c (HbA1c) may improve the glycemic control of people with diabetes by providing a rapid result if the performance of the instruments used is acceptable. A 0.5% HbA1c difference between successive results is considered a clinically relevant change. With this in mind, the In2it from Bio-Rad and the DCA Vantage from Siemens were evaluated according to Clinical and Laboratory Standards Institute (CLSI) protocols.
The CLSI protocols EP-5 and EP-9 were applied to investigate precision, accuracy, and bias. The bias was compared with three certified secondary reference measurement procedures. Differences between capillary and venous blood were investigated by an end-user group consisting of nurse practitioners at a diabetes care center.
At HbA1c levels of 5.1 and 11.2%, total coefficients of variation (CV) for the In2it were 4.9 and 3.3%, respectively, and for the DCA Vantage were 1.7 to 1.8% and 3.7 to 5.5% depending on the lot number of the cartridges. Method comparisons showed significant lot number-dependent results for the In2it and the DCA Vantage compared with the three reference methods. No overall difference was observed between capillary and venous blood for both methods.
Performance results of the In2it and the DCA Vantage showed variable and lot number-dependent results. To maintain the interlaboratory CV of 5% for HbA1c, the Clinical Laboratory Improvement Amendments rules for waived point-of-care instruments should be revised. An obligation for participating in external quality schemes and taking adequate action should be considered for POC instruments that perform poorly.
Hemoglobin A1c (HbA1c), reflecting mean glycemia, is used as a risk parameter for diabetic complications and as a quality assurance indicator for the quality of diabetes care. Point-of-care (POC) instruments for HbA1c are widely used in the world for the measurement of HbA1c. The rapidity of obtaining a result can increase clinical effectiveness and contribute to improved outcomes for patients, but it is imperative that the result provided by the device is accurate and reliable. A faster result is only safe if it is an accurate result. Point-of-care instruments for HbA1c provide relatively quick results and minimize patient inconvenience. Studies have confirmed that immediate feedback of HbA1c levels improves glycemic control in type 1 and insulin-treated type 2 diabetic patients.1–3 Information provided by the manufacturers and limited published data about the performance of POC HbA1c instruments suggest that some of these instruments can compete with clinical laboratory methods.4,5
The aim of this study was to evaluate two POC instruments according to Clinical and Laboratory Clinical and Laboratory Standards Institute (CLSI) protocols under laboratory (CLSI) protocols under laboratory conditions and to discuss the consequences of the findings. The bias of these instruments was compared with three certified International ederation of Clinical International Federation of Clinical Chemistry (IFCC) and or National Glycohemoglobin or National Glycohemoglobin Standardization Program (NGSP) secondary reference measurement procedures, which were calibrated with secondary reference material with assigned IFCC and derived NGSP values and with the mean of the three methods. Moreover, instruments were tested on differences obtained with capillary blood versus venous blood by nurse practitioners at a diabetes care center.
The evaluation consisted of an analytical part by the laboratory and an end-user evaluation by nurse practitioners at a diabetes care center investigating user-friendliness and differences between capillary and venous blood.
The CLSI EP-10 protocol was used to get acquainted with the instruments and to get a general impression of the performances of the instruments.6 The CLSI EP-5 protocol was used to investigate the overall precision (20 days, duplicate measurements twice a day at two levels).7 The EP-9 protocol was used to investigate the bias between the POC instruments and the three different secondary reference measurements procedures (n = 40, duplicate measurements).8 An HbA1c value determination of samples used in the EP-10 and EP-9 protocols was done with two I CC and NGSP certified secondary reference measurement procedures, Roche Tina-quant Gen.2 HbA1c on Integra 800, immunoassay (Roche Diagnostics Ltd., Rotkreuz, Switzerland), Primus Ultra2, affinity chromatography high-performance liquid chromatography (HPLC; Primus Diagnostics, a Trinity Biotech Company, Kansas City, MO), and the certified IFCC secondary reference method Tosoh G7, cation-exchange HPLC (Tosoh Bioscience N.V. S.A., Tessenderlo, Belgium). To check overall calibration and bias, the mean of duplicates of POC instruments in the EP-9 procedure was compared to the mean of the three reference measurements procedures.
An informed consent was obtained from all patients prior to blood collection in accordance with the local ethical committee. Approximately 90% of the measurements was done by two different nurse practitioners, whereas the other 10% was done by three different nurse practitioners. The nurse practitioners were asked about user-friendliness, advantages, and disadvantages of the different POC analyzers.
The two POC HbA1c analyzers evaluated in this study were the DCA Vantage™ (Siemens Medical Solutions Diagnostics, Tarrytown, NY), which is based on inhibition of latex agglutination methodology, providing results in 6 minutes, and the In2it™ (Bio-Rad, Hercules, CA), which is based on affinity chromatography with results available in 10 minutes.
Computations were performed using EP Evaluator Release 8 (David G. Rhoads Associates, PA).9
Table 1 shows precision results of the EP-5 protocol. At HbA1c levels of 5.1 and 11.2%, total coefficients of variation (CV) for the In2It were 4.9 and 3.3%, respectively, and for the DCA Vantage were 1.7 to 1.8% and 3.7 to 5.5% depending on the lot number of the cartridges. Table 2 gives an overview of method comparison results achieved with the EP-9 protocol. The 95% confidence intervals (CI) at medical decision points (MDP) of 6 and 9% HbA1c, respectively, show that the In2it and the DCA Vantage were significantly deviant from any of the three reference methods. To check the overall calibration and bias of the POC instruments, the mean of duplicates of POC instruments were also compared with the mean of the three reference methods (Figures 1A, 1B1B, 2A2A, and 2B2B). These figures show the predicted value (including the 95% CI) at MDP of 6 and 9% HbA1c for the various POC methods.
Results from the DCA Vantage were not within the specifications of the manufacturer. The total CV at high HbA1c values was 5.5% (Table 1). Differences between duplicates seen in the EP-9 protocol with the In2it were also unusual according to the manufacturer. Seven of the 40 samples showed a difference of more than 1.1% absolute at different HbA1c values (mean absolute differences between duplicates for the In2it were 0.52, DCA Vantage 0.21, Ultra2 0.06, Tosoh G7 0.05, Tina-quant 0.08). To rule out particular problems with the lot number used, the EP-9 protocol for both methods was repeated with another lot number. Also the EP-5 protocol was repeated for the DCA Vantage (see Tables 1 and 22). Use of a second lot number diminished the mean difference in duplicates for the In2it from 0.52 to 0.27% absolute HbA1c percentage and remained the same for the DCA Vantage and the reference methods.
No significant difference was found in both methods between capillary and venous blood. The MDP of 6% HbA1c for the In2it was 6.10% (95% CI 5.97 to 6.23%) and for the DCA Vantage was 5.93% (95% CI 5.81 to 6.06%). The MDP of 9% HbA1c for the In2it was 9.11% (95% CI 8.96 to 9.27%) and for the DCA Vantage was 9.07% (95% CI 8.95 to 9.19%).
Nurse practitioners considered both instruments to be user-friendly. The noise produced by the In2it the first 3 minutes and the last minute of the run time was considered as inconvenient and disturbing by one nurse practitioner.
Point-of-care HbA1c instruments are used more and more frequently. So far, the consequences of the introduction of these new types of instruments with their specific characteristics have not been discussed thoroughly in the literature. The evaluation of two types of POC instruments, the In2it and the DCA Vantage, is used here as an example to discuss several important consequences associated with the introduction of POC instruments in this field.
Results of the evaluation of the In2it and the DCA Vantage showed that there is a lot number-dependent performance of both methods. The precision of the In2it expressed in total CV and standard error of estimates in the EP-9 is still a matter of concern. The second lot number showed better results. Unfortunately, one never knows if the precision of a particular lot number is acceptable because no duplicate measurements are run in daily life with POC instruments. The overall calibration of the second lot number for the In2it, as reflected in the overall bias, was acceptable between 6 and 9% HbA1c. Results from the first lot number were influenced by bad duplicates.
The DCA 2000 was one of the first point-of-care instruments and was evaluated in several studies.10–12 Notable is that in all of these studies, results from the DCA 2000 were lower compared with the methods used in the laboratory. Also, a recent evaluation of the DCA Vantage, the successor to the DCA 2000, showed a clear bias but was still considered to have acceptable imprecision and good agreement.13 EP-9 results for the two lots of DCA reagents showed different regression lines. The results were too high (mean bias 0.27) for lot B and slightly low for lot A compared with the mean of the three reference methods and with the individual reference methods. The manufacturer may have overcompensated the calibration of the second lot number in response to results from the first lot number used in this study. rom an analytical point of view, the imprecision of the first DCA Vantage lot at high HbA1c levels was too high (CV was 5.5%) and was not within the specifications of the manufacturer. The second lot number gave better results (total CV was 3.7%).
Apart from point-of-care instruments, interlaboratory variation is still a matter of concern and has stabilized at approximately 5%.14 Holmes and colleagues15 concluded that between-method variability is still a potential source of inaccuracy when HbA1c results are interpreted based on fixed clinical decision thresholds. This is especially the case when POC instruments and laboratory methods are used randomly in the same facility. In order to reduce interlaboratory (interhospital) CV, the NGSP reduced the acceptable bias for manufacturer certification to ±0.85% in 2007 and the College of American Pathologists (CAP) began using the NGSP accuracy grade as the only grading system. In addition, the acceptable total error limit of ±15% was lowered to ±12% and will be reduced further in future CAP surveys.16 By tightening NGSP certification criteria and lowering the acceptable total error limit in the CAP survey (to ±6% by 2011), poor performing methods must improve or will fail to be NGSP certified and some of their users will not pass CAP proficiency testing. Unfortunately, CLIA-waived POC instruments, which sustain part of the interlaboratory CV, are not obliged to join external quality schemes. The end users simply have to follow manufacturer's instructions and may therefore escape from the rules imposed on laboratory methods.17 This is a so-called “hole in the dike.” At one end, proficiency testing criteria will be tightened (laboratory methods) and at the other end there will be no rules or very limited rules for CLIA-waived point-of-care instruments.
The introduction of POC HbA1c instruments in the market will diminish the number of patient samples that are analyzed on one instrument; as a consequence, the Gaussian curve describing HbA1c results within a certain population is expected to get broader even if the performance of the new instruments will be the same as the HbA1c methods used in the laboratory. Point-of-care instruments increase the total number of analyzers per 1000 persons with diabetes. Therefore, interinstrument CV and intercartridge CV are extra sources of variability added to the total CV in comparison to a laboratory method. So far, the current CLSI evaluation protocols do not cover this phenomenon sufficiently.
Results achieved by the NGSP and later on by the I CC working group for the standardization of HbA1c to decrease interlaboratory variability from 20% in 1993 to approximately 5% in 2008 should be supported by adjusting the CLIA-waived rules for HbA1c point-of-care instruments. Annual NGSP manufacturer certification should be done, and every laboratory instrument and every POC instrument should be obliged to join external quality schemes. Adequate actions (improve method or withdrawal from the market) must be administered if the performance of a laboratory and or point-of-care instrument is not acceptable.
The manner in which quality controls are being handled may also need to be redefined. To run a quality control occasionally on POC instruments is adequate because it may tell something about the cartridge used but does not provide any guarantee for the next cartridge. However, the consequences of a bad cartridge may be less severe than a bad reagent in the laboratory (it may involve only one result on the POC instrument versus hundreds in the laboratory). Nevertheless, all POC instruments must be equipped with an electronic check on performance. Moreover, the cartridges need to be equipped with an internal HbA1c control. This may not only be true for POC HbA1c, but in general also applies for other POC tests using separate cartridges. In the end, evaluations of POC instruments must be done by end users. However, if manufacturers are capable of producing cartridges without cartridge-to-cartridge variability, the need for an internal quality control may be less important. To achieve this goal, standards need to be tightened at the level of manufacturers.
Results presented here were obtained by the work of an experienced technician and are therefore likely the best results one can achieve analytically; EP-5 and EP-9 results obtained by less experienced end users may be less precise. Although the use of POC HbA1c instruments has some negative consequences that need to be addressed, it is also important to keep in mind that producing HbA1c results at the time of the patient's visit can improve patient care as well.
Performance results from the In2it and the DCA Vantage showed high variability and lot-dependent results. To maintain the interlaboratory CV of 5% for HbA1c, the rules for CLIA-waived point-of-care instruments should be revised. An obligation for participating in external quality schemes and taking adequate action should be considered for POC instruments that perform poorly.
We thank the nurse practitioners for the POC analyses at the diabetes care center.