|Home | About | Journals | Submit | Contact Us | Français|
Near patient testing (NPT) and point-of-care testing (POCT) using portable benchtop analyzers has become necessary in many areas of the medical community, including biocontainment.
We evaluated the Beckman AcT diff, Abaxis Vetscan HMII (2 instruments), Abbott Cell-Dyn 1800, and Abaxis Vetscan VS2 for within-run precision and correlation to central laboratory instruments using NHP blood.
Compared to the central laboratory instruments, the Beckman AcT diff correlated on 80%; the HMII instruments on 31% and 44%, the CD1800 on 31%, and the VS2 on 71% of assays. For assays with published manufacturers precision guidelines, the AcT diff met all nine, the HMII instruments met one and six of six, and the CD 1800 met one of six.
Laboratories using NPT/POCT must test their individual instruments for precision and correlation, identify assays that are reliable, and exclude or develop supplemental procedures for assays that are not.
Near patient testing (NPT) and point-of-care testing (POCT), often using portable benchtop instruments, has become necessary in many areas of the medical community, to include biocontainment areas. There are a number of concerns that need to be addressed in order for NPT/POCT to provide a beneficial outcome to the clinical care or research conducted. Despite the portability and obvious simplicity of most near patient and point-of-care analyzers, the testing itself still faces preanalytical, analytical, and postanalytical issues just like the testing done in a central lab . Although the analytical aspects of NPT/POCT may be similar to those used in the clinical lab, shifting the testing outside of a well-controlled clinical lab environment, to bedside or clinic presents a unique set of challenges. Specimen quality, inappropriate reagent storage, analysis in hot or humid environments, patient hematocrit and other metabolic conditions that affect accuracy and or precision, and untrained/uncertified personnel are just some of the factors that can affect test results and may go unnoticed. [7, 11]. NPT/POCT typically involves low to moderate instrument costs, yet high individual test costs. Additionally, NPT/POCT is often more expensive than the centralized laboratory equivalent due to the nature of the testing process, loss of economy of scale, and the hidden costs of supervising the test quality. [3, 9].
NPT/POCT devices generate medical information that often leads to clinical action or research results. However, when the device is used inappropriately and incorrect results are produced, the patient is at risk, the costs of care are increased, and the research may be invalid. A standardized training program with verifiable maintenance and calibration records and competency assessment is essential to the quality assurance of NPT/POCT. In an environment where NPT/POCT and central clinical laboratory results will both be utilized, the NPT/POCT results must correlate to the central laboratory results so clinicians and researchers are able to correct their results to the laboratory reference. Comparability and reproducibility are the goals of NPT/POCT quality assurance and the motivation behind regulatory agencies that govern laboratory testing. When a laboratory introduces a new test system, it must verify the accuracy, precision, and reportable range before reporting patient test results. Manufacturer's performance specifications can be used as a guideline, but the laboratory must demonstrate that this verification correlates with its in-house test performance .
Biocontainment facilities present a special challenge for laboratory testing. These facilities have high construction costs, are expensive to operate and maintain, and present unique security concerns. This requires minimizing both space and personnel. As biological samples must remain within containment, testing must also be conducted there. Standard clinical laboratory instruments are large, require a significant amount of space, and necessitate specially trained personnel. A small NPT/POCT unit that could be operated by already existing containment personnel would be of great benefit.
Although there have been reports evaluating NPT/POCT in clinical hematology and biochemistry instruments in human  and veterinary [4, 10] settings, only one involved nonhuman primates (NHP) . We evaluated four hematology and one biochemistry portable benchtop NPT/POCT instruments: Beckman AcT diff (Beckman Coulter, Inc., Fullerton, CA, USA, hematology, AcT diff), Abaxis Vetscan HMII (Abaxis, Union City, CA, USA, hematology, two units: HMII#1; HMII#2), Abbott Cell-Dyn 1800 (Abbott Diagnostics, Abbott Park, IL, USA, hematology, CD1800), and Abaxis Vetscan VS2 (Abaxis, Union City, CA, USA, biochemistry, VS2) and compared them with the appropriate Southwest National Primate Research Center Clinical Pathology Laboratory (SPCL) instrument: Beckman Coulter MAXM (Beckman Coulter, Inc., Fullerton, CA, USA, hematology, MAXM) and Beckman Synchron CX5CE (Beckman Coulter, Inc., Fullerton, CA, USA, biochemistry, BCX5CE). The MAXM and BCX5CE were both validated instruments and served as the gold standard for comparison of the portable benchtop units. The goal of this study was to assess the performance of these portable benchtop analyzers in the field and to educate users of these analyzers to their potential limitations. The two HMII instruments (HMII#1; HMII#2; same model and age) were included to evaluate the benefit of standardization to a single NPT/POCT technology.
The species, sex, age and number of NHP tested and compared using the various hematology and biochemistry instruments are listed in Table 1. Each NHP was considered as identical for testing purposes regardless of species. All NHPs were maintained in an Association for Assessment and Accreditation of Laboratory Animal Care, International accredited facility in individual metal or gang metal and concrete caging. They were fed commercial monkey chow supplemented with grains, fruits and vegetables. Water was supplied ad libitum. All animal care and procedures were approved by the Southwest Foundation for Biomedical Research Institutional Animal Care and Use Committee. NHPs were sedated using a single intramuscular injection of ketamine HCl (5-10 mg/kg) (KetaVet, Phoenix Scientific, St. Joseph, MO, USA). Blood samples were collected conventionally in vacutainer tubes (BD, Franklin Lakes, NJ, USA) through femoral venipuncture.
All assays were performed in the SPCL by or under the direction of the SPCL registered medical technologists (CLS, MCR, and RLS). Hematology data was obtained from blood placed in EDTA tubes. For biochemistry, blood was collected into lithium heparinized tubes. The blood was centrifuged and the plasma removed from the cells within 1 hour of collection. Samples were stored at room temperature and tested on all instruments within 4 hours. Paired assays used for correlation were performed on samples from the same vacutainer. As the SPCL does not routinely perform amylase assays, an aliquot of plasma was frozen from 55 NHP and sent to a commercial laboratory (Scott & White Reference Laboratory, Temple, TX, S&W) for amylase determination. In order to broaden the range of biochemical data, the clinical records database (EZPro, GloMed Systems Inc., North Richland Hills, TX, USA) was utilized to identify previously analyzed stored NHP serum samples with more extreme results. Fourteen archived frozen serums (collected as described above and that had been stored at −20°C) were identified, quickly thawed, and analyzed.
Calibration, reproducibility and carryover studies are performed in the SPCL quarterly on the MAXM. Calibrations for the BCX5CE are done based on individual reagent frequency specifications. For the MAXM and the BCX5CE, three levels of controls are run daily, and results are submitted monthly for peer comparison by the Beckman Interlaboratory Quality Assurance Program (Beckman Coulter, Inc., Fullerton, CA, USA). The benchtop units were validated and calibrated using control assay materials. After all SPCL and benchtop units met their respective control and calibration requirements, normal patient specimens were utilized for within-run precision and correlation studies.
A total of 60 (51 Macaca fascicularis (MF) and 9 Macaca mulatta (MM)) assays were run in parallel using the HMII#1, HMII#2, and MAXM. The two HMII instruments were the same model and age. Since both would be used during research studies, it was important to correlate them with each other in addition to the SPCL MAXM. A total of 42 (20 Papio sp (PS), 18 MM, 3 Pan troglodytes (PT), and 1 MF) assays were run in parallel using the CD1800 and MAXM. A total of 62 (48 PS, 7 MM, 6 PT, and 1 MF and) assays were run in parallel using the AcT diff and MAXM.
A total of 69 (56 MF and 13 MM) assays were run in parallel using the VS2 and the BCX5CE. Fourteen (1 MF and all 13 MM) of these were frozen archived samples; the other 55 were heparinized plasma. Aliquots from the 55 heparinized plasma samples (all MF) were sent for amylase determination by a commercial laboratory.
All five NPT/POCT instruments were tested for within-run precision to estimate the expected daily performance [1, 8]. A randomly selected sample was assayed eleven times and one outlier result was excluded. The remaining 10 results were evaluated. Mean, standard deviation and coefficient of variation were calculated for each parameter.
All five NPT/POCT instruments were correlated to the appropriate SPCL laboratory instrument or to the amylase results from the commercial laboratory. The Pearson correlation test was run on each dataset and the correlation coefficients (r value) determined. Assays with a correlation coefficient of less than 0.95 were considered unacceptable . Data analysis was performed using Analyse-It Software v.2.08 (Analyse-It Software Ltd., UK).
Results of the within-run precision testing are shown in Tables Tables22 and and3.3. Reproducibility varied considerably between instruments. Where available, results were compared with each manufacturer's published precision guidelines. The AcT diff was within the manufacturers guidelines for nine of nine assays. One of the HMII instruments was within manufacturers guidelines for six of six assays; the other HMII for only one of six assays. Interestingly, the values obtained for lymphocytes and monocytes (both number and percent) appeared reversed in two of the 11 consecutive analyses of the same sample using the HMII#1 unit. The CD 1800 was within the manufacturers guidelines for one of six assays. Manufacturer's published precision guidelines were unavailable for the VS2; the precision guidelines for the BCX5CE are listed for comparison.
The correlation coefficient results for the NPT/POCT instruments are shown in Tables Tables44 and and5.5. Each NPT/POCT device performed multiple individual assays. Each assay was evaluated for achieving acceptable correlation to the SPCL instruments. Additionally, the two HMII units (HMII#1 and HMII#2) were compared for correlation to each other.
The Beckman AcT diff correlated with the MAXIM for 80% of the assays: WBC, RBC, HGB, MCV, MCH, RDW, PLT, MPV, GRAN%, GRAN#, LYM%, and LYM#. It did not correlate for: MCHC, MONO%, and MONO#.
The HMII #1 correlated with the MAXIM for 44 % of the assays: WBC, RBC, HGB, MCV, MCH, GRAN%, and GRAN#. It did not correlate for: HCT, MCHC, RDW, PLT, MPV, LYM%, LYM#, MONO%, and MONO#.
The HMII #2 correlated with the MAXIM for 31% of the assays: WBC, MCV, MCH, Gran%, and Gran#. It did not correlate for: RBC, HGB, HCT, MCHC, RDW, PLT, MPV, LYM%, LYM#, MONO%, and MONO#.
The Abbott Cell-dyn 1800 correlated with the MAXIM for 31% of the assays: WBC, HGB, MCV, PLT, and LYM#. It did not correlate for: RBC, HCT, MCH, MCHC, RDW, MPV, GRAN%, GRAN#, LYM%, MONO%, and MONO#.
The HMII #1 correlated with the HMII #2 for 63 % of the assays: WBC, MCV, MCH, RDW, GRAN%, and GRAN#. They did not correlate for: RBC, HGB, HCT, MCHC, PLT, MPV, LYM%, LYM#, MONO%, and MONO#.
The VS2 correlated with the BCX5CE for 71% of the assays: GLU, BUN, CREA, TP, ALB, ALT, ALP, K, PHOS, and AMY. They did not correlate for: GLOB, NA, CA, and TBIL.
We compared the test results of five NPT/POCT hematology and biochemistry instruments with the SPCL instruments. There was more variance in the results than we expected. The AcT diff performed the best for reproducibility. Although the HMII#2 unit also performed well for reproducibility, the poor performance of the HMII#1 unit makes any conclusion concerning this instrument difficult. It is also important to realize that many of the individual assays, and the VS2 unit overall, did not have any published manufacturer's standards for reproducibility. While running more replicates would possibly have improved the reproducibility results, reproducibility is an automated procedure on the MAXM, the ACT diff, and the CD1800. All three instruments require eleven tests of the sample, delete one outlier, and then calculate the reproducibility. For consistency, we chose to use this methodology where manufacturer's standards for reproducibility were unavailable.
There were serious shortcomings for correlation results, with 3 of the hematology NPT/POCT instruments achieving acceptable correlations on only 31-44% of assays. The Beckman AcT diff performed much better, achieving acceptable correlation on 80% of the hematology assays. The single chemistry NPT/POCT achieved an acceptable correlation on 71% of the biochemistry assays.
Several factors can influence NPT/POCT test results, including specimen quality, inappropriate reagent storage, analysis in hot or humid environments, patient hematocrit and other metabolic conditions that affect accuracy and or precision, and untrained/uncertified personnel.  As the samples in this report were analyzed contemporaneously in the SPCL on blood samples from the same vacutainer tube by registered medical technologists, it is unlikely that these factors were involved. We believe that the results reported here are due to the NPT/POCT instruments themselves.
Clinical laboratory instruments report results for both assays that directly or indirectly measure an element of the blood and also for assays that are calculated by the instrument based on the results of those measured assays. Generally correlation between calculated assays was less than that from the measured assays they were derived from. This was not unexpected as it would be extremely unlikely that random error from two measured results would occur in such a way as to cancel out the effect. It would be expected that, as seen here, combining measured values would increase the amount of random error in the final product.
Standardization to a single NPT/POCT technology has been suggested as the most important step to improve NPT/POCT quality . While this decreases the number of different devices and differing methodologies within an organization, the poor correlation (only 63% of the assays) seen between the two HEMII hematology units demonstrates that standardization of instruments alone may not be sufficient to insure adequate results. Why the correlations between two instruments of the same model were so low is unknown. Portable benchtop units are calibrated at the factory and not designed to be modified at the user level.
A biocontainment environment has complex and restricted access. There was a limited window during which these machines were available for evaluation. This resulted in some limitations. First, was combining both the lithium heparin plasma samples and serum samples for evaluating the correlation of the chemistry units. Typically each type of fluid is correlated separately. Analyzing the archived serum samples with the plasma samples was required to insure an adequate range of biochemical data results for performing the correlation calculations. Second was the use of multiple species within each assay. Not all species were tested on each instrument and samples from all species were not received in the laboratory during the times that each instrument was available for evaluation. Although species are not typically combined for correlation studies, it is not relevant in this instance as the correlation between the machines was evaluated, not determining a specific species normal value.
Despite the shortcomings identified with these instruments, limited utilization of these instruments with supplemental procedures is possible in biocontainment areas where access to a central lab is not possible. The results of the assays that failed to meet the 95% correlation requirement could be withheld and other methodologies utilized to obtain that data. The problems with the white blood cell differentials in the hematology units tested here could be supplemented by performing concurrent manual differentials on the blood samples in conjunction with the total WBC value attained from the instrument. This would provide an accurate count of lymphocytes, monocytes, and granulocytes, as well as an estimate of platelet number. Likewise a microhematocrit centrifuge would supply the missing hematocrit data.
Laboratories using NPT/POCT instruments must be aware of precision and correlation issues and evaluate the individual machines in their facilities to identify the particular assays that are reliable and those assays that are unreliable. Assays identified as unreliable should not be reported or utilized and alternate methodologies identified for obtaining that information.
The authors acknowledge the assistance Ms Stacey Perez and the biocontainment facility veterinary technical staff for assisting with the testing procedures and of Ms Pricilla Williams and Dr Mark Sharp for statistical support. All authored materials constitute the personal statements of David L. McGlasson and are not intended to constitute an endorsement by the U.S. Air Force or any other Federal Government entity This work was supported by the National Institutes of Health/National Center for Research Resources (NIH/NCRR) grant P51 RR013986 to the Southwest National Primate Research Center. NHP were housed in facilities constructed with support from Research Facilities Improvement Program Grant C06 RR016228 from the NIH/NCRR.