|Home | About | Journals | Submit | Contact Us | Français|
Electronic health record (EHR) systems are often modified through the addition of new features over time. Few studies have examined the specific effects of such changes. We examined whether implementation of a bidirectional laboratory interface for order entry and data reporting within an existing ambulatory EHR would result in more prompt responses to laboratory indications for antiretroviral therapy (ART) changes or in improved communication with HIV+ patients about relevant laboratory results.
We conducted a single-arm intervention study comparing the timeliness of ART regimen changes, HIV viral load (VL) outcomes and patient-reported assessments of care before and after implementation of a laboratory data exchange interface within an existing EHR, without changing the EHR ordering or results reporting user interface. Patient data was extracted from the EHR covering the period from 1 year before to 2 years after the intervention for a cohort of 1181 patients who had received care during the baseline year. The timeliness of ART changes was represented by the days from a laboratory-result “signal” (CD4 dropping below 350 or 200 or VL increasing by a half-log or to a value over 100,000) to an ART-change “response.” Patient assessments of care were collected by interviewing 100 anonymous patients at baseline and another 125 at 2 years post-intervention.
A total of 171 laboratory “signal” events were followed within 80 days by a change in ART therapy. The mean time from signal to therapy change (adjusted for clustering by patient) initially increased, from 37.7 days during the pre-intervention year to 48.2 days during the quarter immediately following activation of the lab intervention. It then declined to a mean of 31.4 days over the remaining 21 months of observation (P=0.03 for the 6-day improvement from the pre-period). A majority of patients (65%) achieved undetectable VL values by the end of the observation period; faster signal-response times were not associated with greater achievement of undetectable VL. Patients rated communication about laboratory tests more highly after implementation of the interface (91 vs. 83 on a 100-point scale, P=0.01); ratings were not higher for other aspects of care.
Adding laboratory data exchange interfaces within existing EHRs holds promise for improving HIV care, both in the timeliness of responses to important laboratory results and in the quality of provider communication about lab tests. However, the benefits from this incremental change may be modest unless more extensive redesign of laboratory follow-up workflows is undertaken, with support from enhanced user interfaces that take advantage of the laboratory information delivered. Providers should also consider increased staffing to compensate for dips in follow-up performance during the initial post-implementation months.
Clinical laboratory tests are a cornerstone of health care, but many physician offices have unreliable systems for managing test results, including those having electronic health records (EHRs) [1–3]. Surveys of EHR users have found that most are dissatisfied with their capacity to manage abnormal laboratory results  and that they often experience delays in becoming aware of significant abnormalities . Electronic interchange of test orders and results between EHRs and clinical laboratories is an important element for ensuring that the correct tests are obtained and that results are followed up for all ordered tests . Exchanging laboratory data is one aspect of health information exchange, a topic which is now being actively investigated [7–9], and these general investigations are beginning to include the exchange of data specific to HIV care . However, few studies have focused specifically on the impact of data exchange between clinical laboratories and EHRs. One study from Denmark found that by eliminating the need to hand-enter both orders and results such a system reduced the error rate by a factor 10 and reduced reporting time by 2 days . This reduced time should increase the probability of timely follow-up and may in some cases reduce the need for additional appointments when results are not available.
Laboratory results are particularly important in HIV care. In the last 10–12 years, several new classes of life-saving antiretroviral treatment (ART) have come into routine clinical use, and it is widely recognized that with unprecedented speed they have transformed HIV from a rapidly fatal illness into a manageable chronic illness. However, in order for the more than 1 million persons living with HIV in the U.S. to benefit from these treatment advances, timely viral load (VL) and CD4 test results are required to determine the need for ART and to monitor patients for response to treatment . In addition to being life-extending, timely treatment also decreases the risk of HIV transmission [13–15]. Thus, laboratory data exchange could have significant benefits for the care of individual HIV patients and also for the broader public health goal of preventing forward transmission of the virus .
In the current study, our goals were to examine the effects of implementing clinical laboratory data exchange within an EHR on the timeliness of adjustments in ART therapy and on patient reports of communication related to laboratory test results among a cohort of HIV+ patients receiving HIV care in a community clinic.
Our study site was the Comprehensive AIDS Resource Education (CARE) clinic at St. Mary’s Medical Center, Long Beach, California. CARE is the largest provider of HIV services in the city of Long Beach, serving an entirely-HIV+ patient population. Patients are primarily male (83%), and their primary risk factor is being men who have sex with men (MSM) (68%). The racial-ethnic mix of patients is diverse, including people of African American (18%), Hispanic (28%), and White (43%) ethnicity or race. Twenty-nine percent of patients are uninsured and 21% have Medicaid. The CARE clinic implemented the NextGen EHR and practice management system in 2001. The system has been maintained and continuously upgraded.
Prior to the intervention, physicians entered lab test orders into the EHR during visits and then printed paper requisitions for patients to hand-carry to the phlebotomist. For most tests, prior to collecting specimens the phlebotomist entered the orders into a computer terminal provided by the contracted clinical laboratory (LabCorp). When a need for future testing could be predicted, patients were given orders to have tests done 2–3 weeks prior to a future follow-up appointment. However, patients often failed to comply with advance testing, in which case the tests would be done at the follow-up visit, with telephone follow-up planned for abnormalities.
Test results were returned by fax, and fax pages were screened by a physician. When abnormalities potentially needing action were identified, patients were contacted by staff to schedule an additional follow-up visit to assess adherence and consider therapy changes. Results were manually entered into the EHR by a staff member who also handled patient scheduling and referrals. The data-entry backlog for labs varied from 1–4 weeks, depending on the staff person’s availability. At follow-up visits, if the provider found that expected results were not in the EHR, then medical assistants helped to retrieve the faxed results from the data-entry queue.
As part of a larger national demonstration project [17, 18], CARE received funds to enhance the clinic’s data management system by establishing a bidirectional lab interface. After the intervention, lab ordering didn’t change for the physician but the orders were transmitted directly to LabCorp. Phlebotomists then used a computer printout to guide specimen collection rather than entering the orders manually. The policy of obtaining tests 2–3 weeks in advance of future visits, when appropriate, did not change.
Test results were returned electronically into the EHR without needing to be data-entered. However, physicians continued to use faxed results to screen for abnormalities, and they used marked-up fax pages to task staff with scheduling follow-up visits, in the same manner as before the intervention. This manual process was retained because the EHR’s interface for reviewing lab results required substantial scrolling and paging to see all of a given patient’s results, did not highlight abnormalities, did not show reference ranges, and lacked a function to task staff with scheduling follow-up for abnormalities. Physicians perceived that the lab interface required about 1 minute per patient vs. less than 10 seconds per patient for reviewing and routing the printed results. Thus, after the intervention, the primary changes in work processes were the elimination of the results data entry and the elimination of results pursuit from the data-entry queue when patients returned for follow-up visits. The staff time saved went primarily to enhanced patient telephone contact.
We created a patient cohort consisting of the 1181 patients who had any encounter during the year prior to implementation of the bidirectional lab interface (12/1/2007 to 11/30/2008). Encounters included (1) face-to-face visits with a physician, nurse practitioner, registered nurse, social worker, dietician or adherence counselor, (2) injections, (3) TB checks, (4) lab draws, (5) telephone calls that resulted in an EHR note, (6) medication refills, (7) chart updates (lab, diagnostic results, etc) or (8) sub-specialty referrals. For this cohort, information systems personnel at the clinic then extracted and de-identified data from the NextGen EHR covering the time period from 12/1/2007 through 11/30/2010. The data extracted included patient demographics, insurance and other characteristics collected for purposes of regulatory reporting (income, homeless status), provider visit dates, laboratory test dates and values, and prescriptions and prescription dates. In the data used for all analyses, patients were identified only by a study ID that was assigned during the de-identification process and all patient-identifying fields were deleted.
We conducted anonymous cross-sectional interviews in November 2008 and February 2011 with consecutive patients recruited from the clinic waiting room. Patients were not approached for recruitment; rather a sign posted at the registration desk announced the interview opportunity and patients had to request participation. Interviews were conducted using computer-assisted personal interviewing software (CAPI) except for questions regarding sensitive content areas (sexual behaviors, drug use), for which audio computer-assisted self-interview (ACASI) modules were used. The interview instrument was designed based on a review of the literature and consultation with experts involved with the Consumer Assessment of Healthcare Providers and Systems survey (CAHPS) . The final instrument included the CAHPS 4-item general communication scale (how often providers listen carefully, explain things in an understandable way, show respect for what respondents had to say, and spend enough time with respondents), two overall satisfaction items from CAHPS (rating overall quality of care, recommendation of place to others), and four provider trust items taken from two sources [20, 21] (trust HIV care team to offer high quality care, trust HIV care team to put your medical needs above all other considerations, doctor will do whatever it takes to get me all the care I need, doctor only thinks about what is best for me). Based on the wording of CAHPS communication items, we also created 4 new items assessing the quality of communication specifically about lab tests, including items that assessed how often providers explained the tests ordered in a way that was easy to understand, how often providers explained the results of tests in a way that was easy to understand, how often providers explained the results of viral load tests in a way that was easy to understand, and how often providers explained the results of CD4 count tests in a way that was easy to understand.
We constructed variables representing intensifications of the patient’s ART regimen (including the initiation of ART) and changes in laboratory values that could constitute a “signal” for changes in disease control that might warrant initiation or adjustment of ART : increases in VL by a half-log (relative) or from below to above 100,000 copies/ml (absolute value), or decreases in CD4 count from above to below 350 or from above to below 200. These signals were classified as having a regimen-change response if they had a new ART medication (at the ingredient level) prescribed within 1–80 days after the signal. The 80-day threshold was selected by reviewing with the clinic’s lead physician cases of ART regimen changes that took place within 61 to 90 days of a signal. For all cases having a delay time less than 80 days, the regimen change appeared to be a response to the signal, whereas regimen changes that took place more than 80 days after the signal appeared to be more likely due to other factors such as side effects or subsequent laboratory changes.
To assess whether good disease control was obtained after medication changes made in response to signals, we constructed a variable representing whether the patient had achieved an undetectable VL result within the observation period and before any subsequent medication change.
We used Chi-square or Fischer exact tests to assess bivariate associations among categorical variables and t-tests or ANOVA tests to compare the values of continuous variables between groups. To explore associations of patient characteristics with signal-response times we used multivariate linear regression. Statistical calculations were carried out in SAS version 9.2 (SAS Institute, Cary, NC). All signal-response time and viral load analyses were adjusted for clustering of observations within patient using Taylor series variance estimation in the SAS SURVEYMEANS and SURVEYREG procedures. All reported P values are 2-tailed. The study was approved by the institutional review boards both at UCLA and St. Mary’s Medical Center.
Over the 3-year observation period, 547 patients (46% of the 1181-patient cohort) had a total of 1093 sets of laboratory results that qualified as a “signal” possibly warranting a change in therapy. Among these signal events, there were 171 (among 144 patients) that were followed by a change in the patient’s ART regimen within 80 days. Anecdotally, the majority of laboratory signal events were due to identifiable interruptions in medication adherence, for example associated with substance use relapses. Efforts for these patients focused on adherence rather than medication changes. Other factors accounting for signals with no response included patients known to have multidrug resistant HIV, patients declining regimen changes, and changes that constituted clinically insignificant fluctuations of CD4 or VL values around the usual treatment thresholds. As shown in Table I, the subset of patients having a signal-response pair tended to have less private insurance and greater use of ambulatory visits, adherence counseling, HAART therapy, case management visits, and mental health services than those who had no signal-response pair, but they were otherwise similar in all demographic characteristics.
Table II shows details for each type of signal, including its frequency, frequency of response, and response examples. Of note, many signal events were co-occurrences of more than one type of change.
For the 171 signal-response pairs, the median time from signal to response (the prescription of a new ART medication) was 34 days (range 1–78, inter-quartile range 18–54, mean 36.1). After adjustment for clustering by patient, the overall mean response time was 35.9 days. Figure 1 charts the adjusted mean response time by quarter over the 3-year observation period. Of note, the response time jumped from an adjusted mean of 37.7 days during the pre-intervention year to 48.2 days during the quarter immediately following activation of the lab intervention (an increase of 10.5 days, 28%). It then declined to a mean of 31.4 days over the remaining post-intervention period (a decrease of 6.3 days, 17% from the pre-period, P=0.03). No patient characteristics were significantly associated with the response time.
Overall, patients had an average of 0.97 follow-up provider visits after the signal date and before the response prescription date. This average was 1.15 before the intervention, 1.19 during the implementation quarter, and 0.73 for the remaining post-intervention period (a 37% decrease vs. the pre- period, P=0.10). Forty four percent of signal events occurred on a date with no provider visit, implying that the patient came in for testing prior to a visit, per the clinic’s desired protocol. The average response time for these cases was 33.5 days vs. 37.8 days for others (P=0.19 for the 11% difference). By time period, this pre-visit testing rate was 40% before the intervention, 31% during the first post-implementation quarter, and then 51% for the remaining period (P=0.28). We also found that response prescriptions occurred on dates with no visit in 31% of cases, demonstrating that prescriptions were sometimes called in (not the clinic’s desired protocol); this proportion was 26% before, 38 % during, and 35% after (P=0.49). Finally, the average time between follow-up events (follow-up visits between signal and response plus the response itself, whether or not it occurred at a visit) increased from 21 to 29 days during the first quarter after the intervention, and then it returned to 21 days over the remaining observation period (a 38% increase during the first post-implementation quarter, P=0.07).
For 23 of the 171 signal-response pairs (13%), the signal involved a change in CD4 count only along with a VL that remained undetectable; these were excluded as not informative for the analysis of achieving undetectable VL. Another 11 signal-response pairs (6.4%) were excluded from this analysis because there were no VL lab results during the observation period subsequent to the ART regimen change. Among the remaining 137 medication-change events that were made when the patient had a detectable VL, 89 (65%) were followed by achievement of an undetectable VL during the observation period and before any subsequent medication change, whereas 48 (35%) never achieved an undetectable VL during this time. As shown in Figure 2, the probability of achieving an undetectable VL was not associated with the signal-response time. The mean signal-response delay was 35.5 days for medication changes that were followed by the patient achieving undetectable status vs. 36.3 days for changes that were followed by only detectable viral loads.
Table III compares the cross-sectional patient survey responses before and after implementation of the lab interface. There were statistically significant differences in respondent race, age, and HIV risk factor, but not in other characteristics. The differences were relatively small, for example a 3-year difference in mean age. After implementation of the laboratory interface, patients rated communication about laboratory tests significantly higher, on 3 of 4 individual items and when these items were combined into a scale score. By comparison, patients did not rate general communication or trust in their doctors and HIV team more highly. There was a trend toward higher ratings on the overall satisfaction scale after the lab interface but these did not achieve the conventional level of statistical significance for the scale or for either of the component questions individually.
Implementation of the bi-directional lab interface was associated with a modest but statistically significant improvement in the time required for the clinic to enact regimen changes after important HIV-specific changes in laboratory test results. This response time represents time during which the HIV disease is not in control. The improvement of one week in this delay could be clinically important—for reducing the potential for forward transmission of HIV in addition to reducing the direct health effects of HIV and the risk of opportunistic infection for the individual patient. Because delays in therapy could increase the potential for mutations to result in resistant strains, we had hypothesized that longer signal-response times might decrease the probability of achieving an undetectable viral load. One prior study found that delay in ART initiation was associated with failure to recover normal CD4 cell counts. Our results did not support this hypothesis, but the incremental delays in our study were shorter than those documented in the prior study.
The improvement we observed in response times was mediated by a decrease in the number of follow-up visits between signal and response. In the present study, implementation of the lab interface resulted in only a few direct process changes that could have affected the clinic’s response times. First, phlebotomists no longer transcribed orders, eliminating a potential source of error in the tests obtained. Such errors could have increased response-time if there were errors in follow-up testing after the initial signal (to recheck control after an adherence intervention or to check viral genotype for resistance). No data was available on the frequency of errors in test-ordering, but occasional errors in ordering genotype or other follow-up tests after a signal could have led to some additional follow-up visits. Probably a greater source of reduction in follow-up visits, however, was achieved through eliminating the queue of un-entered results. Test results were occasionally unavailable in the EHR when patients returned for follow-up, creating disruption for physicians and for staff who were tasked with finding the printed results in such cases. Unavailable results probably led to some additional visits. In addition, the staff time freed up from pursuing results may have enabled better coordination for obtaining lab tests in advance of future clinic visits. Although the 20% improvement we observed in the use of pre-visit testing wasn’t statistically significant, these incidents were associated with slightly shorter response times. The increased availability of results in the EHR probably also explains the improvement that patients noticed in communication about laboratory tests.
The fact that clinicians continued to screen incoming results on fax pages, in essence working around the EHR's features for this, indicated a need for better EHR usability and matching of EHR design with clinical workflows. Recent efforts to decompose the tasks of laboratory result management represent an important step toward the broader goal of improving EHR support for these tasks [3, 23] These efforts will need to be paired with ongoing evaluation and monitoring of delays in test result follow-up . Studies such as the present one, evaluating incremental changes in EHR systems, will also be increasingly important as EHR adoption spreads.
Finally, we observed a common pattern of change in performance after implementation of a system change—an initial dip in performance followed by improved performance over subsequent time periods. Although this phenomenon is familiar to implementers, the health information technology (IT) literature contains more reports of outright implementation failures (such as one detailed report of implementation failure for a laboratory order-entry system ) than reports of temporary performance setbacks due to EHR adoption. We are aware of only one article documenting this phenomenon around EHR implementation , and we were unable to find any additional reports in searching Pubmed. Furthermore, we were able to find only a few studies in the broader implementation literature that deal formally with this performance-dip phenomenon (e.g., ). More commonly, studies evaluating the impact of health IT interventions deal with this phenomenon by simply excluding the initial months or even a year after the implementation. For example, a study of response times to abnormal potassium results before and after EHR implementation excluded an entire year following EHR implementation . More research is needed to understand and plan for minimizing these performance setbacks, which can be substantial, as we experienced. Although we do not have direct evidence for the mechanism underlying this effect, we did find that the time between follow-up events was 39% longer in the post-implementation quarter than in the pre-intervention or remaining post-intervention periods. This additional delay was probably due to increased workloads for scheduling and telephone staff during the transition time when some lab results were available immediately in the EHR system while older results (and a few results from non-connected labs) still remained in the data-entry queue, which was still being worked down by the same staff. These results further suggest that dips in follow-up performance might be remediable through hiring temporary staff or offering over-time during the vulnerable period to staff responsible for patient scheduling and telephone follow-up.
In conclusion, adding electronic laboratory data exchange in HIV care holds promise for improving the timeliness of important therapy changes and for improving patient-centered communication about laboratory tests. However, more extensive process re-engineering, supported by more substantial redesign of EHR user interfaces, may enable greater benefits to be realized from laboratory data exchange. Provider organizations should also plan to compensate for dips in follow-up performance during the initial months after implementation of new HIT capabilities.
|What was already known on this topic||What this study added to our knowledge|
This research was supported by a grant from the Health Resource and Services Administration (HRSA-07-046). Dr. Cunningham received partial support for his time on this study from the NIDA (R01 DA030781), and the NIMH (R34 MH089719). Drs. Bell and Cunningham Mr. Nakazono had partial support from the National Center for Advancing Translational Science (UL1TR000124). The funders had no role in the analysis and interpretation of data or in the writing of the manuscript. We are grateful to Jimmy Ngo for assistance in preparing the manuscript.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Authors’ contributionsEach of the authors (Bell, Cima, Seiden, Nakazono, Alcouloumre, and Cunningham) contributed to the conception and design of the study, to interpretation of data, and to drafting or critically revising the article and to final approval of the version submitted.
Conflict of interest statement
None of the authors has any financial or personal relationships with people or organizations that could inappropriately influence bias or bias the work reported in this paper.