PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Int J Med Inform. Author manuscript; available in PMC Oct 1, 2013.
Published in final edited form as:
PMCID: PMC3477872
NIHMSID: NIHMS401975
Effects of Laboratory Data Exchange in the Care of Patients with HIV
Douglas S. Bell, M.D., Ph.D.,1 Laral Cima, M.S.,2 Danielle Seiden, M.P.P.,1 Terry Nakazono, M.S.,1,3 Marcia Alcouloumre, M.D.,2 and William E. Cunningham, M.D., MPH1,4
1Division of General Internal Medicine and Health Services Research, Department of Medicine, David Geffen School of Medicine at UCLA
2St. Mary’s Medical Center., Long Beach, CA
3UCLA Health System Patient Safety Institute, UCLA School of Nursing
4Department of Health Services, UCLA School of Public Health
Correspondence to: Douglas S. Bell, MD, PhD, Division of General Internal Medicine, David Geffen School of Medicine at UCLA, 911 Broxton Ave., Rm. 227, Los Angeles, CA 90024. (310) 794-3168 • FAX (310) 794-0732, dbell/at/mednet.ucla.edu
Background
Electronic health record (EHR) systems are often modified through the addition of new features over time. Few studies have examined the specific effects of such changes. We examined whether implementation of a bidirectional laboratory interface for order entry and data reporting within an existing ambulatory EHR would result in more prompt responses to laboratory indications for antiretroviral therapy (ART) changes or in improved communication with HIV+ patients about relevant laboratory results.
Methods
We conducted a single-arm intervention study comparing the timeliness of ART regimen changes, HIV viral load (VL) outcomes and patient-reported assessments of care before and after implementation of a laboratory data exchange interface within an existing EHR, without changing the EHR ordering or results reporting user interface. Patient data was extracted from the EHR covering the period from 1 year before to 2 years after the intervention for a cohort of 1181 patients who had received care during the baseline year. The timeliness of ART changes was represented by the days from a laboratory-result “signal” (CD4 dropping below 350 or 200 or VL increasing by a half-log or to a value over 100,000) to an ART-change “response.” Patient assessments of care were collected by interviewing 100 anonymous patients at baseline and another 125 at 2 years post-intervention.
Results
A total of 171 laboratory “signal” events were followed within 80 days by a change in ART therapy. The mean time from signal to therapy change (adjusted for clustering by patient) initially increased, from 37.7 days during the pre-intervention year to 48.2 days during the quarter immediately following activation of the lab intervention. It then declined to a mean of 31.4 days over the remaining 21 months of observation (P=0.03 for the 6-day improvement from the pre-period). A majority of patients (65%) achieved undetectable VL values by the end of the observation period; faster signal-response times were not associated with greater achievement of undetectable VL. Patients rated communication about laboratory tests more highly after implementation of the interface (91 vs. 83 on a 100-point scale, P=0.01); ratings were not higher for other aspects of care.
Conclusions
Adding laboratory data exchange interfaces within existing EHRs holds promise for improving HIV care, both in the timeliness of responses to important laboratory results and in the quality of provider communication about lab tests. However, the benefits from this incremental change may be modest unless more extensive redesign of laboratory follow-up workflows is undertaken, with support from enhanced user interfaces that take advantage of the laboratory information delivered. Providers should also consider increased staffing to compensate for dips in follow-up performance during the initial post-implementation months.
Keywords: HIV Infections/prevention & control, Computer Communication Networks, Ambulatory Care Information Systems, Clinical Laboratory Information Systems, Medical Order Entry Systems, Electronic Health Records, Communication, Evaluation Studies
Clinical laboratory tests are a cornerstone of health care, but many physician offices have unreliable systems for managing test results, including those having electronic health records (EHRs) [13]. Surveys of EHR users have found that most are dissatisfied with their capacity to manage abnormal laboratory results [4] and that they often experience delays in becoming aware of significant abnormalities [5]. Electronic interchange of test orders and results between EHRs and clinical laboratories is an important element for ensuring that the correct tests are obtained and that results are followed up for all ordered tests [6]. Exchanging laboratory data is one aspect of health information exchange, a topic which is now being actively investigated [79], and these general investigations are beginning to include the exchange of data specific to HIV care [10]. However, few studies have focused specifically on the impact of data exchange between clinical laboratories and EHRs. One study from Denmark found that by eliminating the need to hand-enter both orders and results such a system reduced the error rate by a factor 10 and reduced reporting time by 2 days [11]. This reduced time should increase the probability of timely follow-up and may in some cases reduce the need for additional appointments when results are not available.
Laboratory results are particularly important in HIV care. In the last 10–12 years, several new classes of life-saving antiretroviral treatment (ART) have come into routine clinical use, and it is widely recognized that with unprecedented speed they have transformed HIV from a rapidly fatal illness into a manageable chronic illness. However, in order for the more than 1 million persons living with HIV in the U.S. to benefit from these treatment advances, timely viral load (VL) and CD4 test results are required to determine the need for ART and to monitor patients for response to treatment [12]. In addition to being life-extending, timely treatment also decreases the risk of HIV transmission [1315]. Thus, laboratory data exchange could have significant benefits for the care of individual HIV patients and also for the broader public health goal of preventing forward transmission of the virus [16].
In the current study, our goals were to examine the effects of implementing clinical laboratory data exchange within an EHR on the timeliness of adjustments in ART therapy and on patient reports of communication related to laboratory test results among a cohort of HIV+ patients receiving HIV care in a community clinic.
Study Population and Setting
Our study site was the Comprehensive AIDS Resource Education (CARE) clinic at St. Mary’s Medical Center, Long Beach, California. CARE is the largest provider of HIV services in the city of Long Beach, serving an entirely-HIV+ patient population. Patients are primarily male (83%), and their primary risk factor is being men who have sex with men (MSM) (68%). The racial-ethnic mix of patients is diverse, including people of African American (18%), Hispanic (28%), and White (43%) ethnicity or race. Twenty-nine percent of patients are uninsured and 21% have Medicaid. The CARE clinic implemented the NextGen EHR and practice management system in 2001. The system has been maintained and continuously upgraded.
Pre-Intervention Lab Test Management Process
Prior to the intervention, physicians entered lab test orders into the EHR during visits and then printed paper requisitions for patients to hand-carry to the phlebotomist. For most tests, prior to collecting specimens the phlebotomist entered the orders into a computer terminal provided by the contracted clinical laboratory (LabCorp). When a need for future testing could be predicted, patients were given orders to have tests done 2–3 weeks prior to a future follow-up appointment. However, patients often failed to comply with advance testing, in which case the tests would be done at the follow-up visit, with telephone follow-up planned for abnormalities.
Test results were returned by fax, and fax pages were screened by a physician. When abnormalities potentially needing action were identified, patients were contacted by staff to schedule an additional follow-up visit to assess adherence and consider therapy changes. Results were manually entered into the EHR by a staff member who also handled patient scheduling and referrals. The data-entry backlog for labs varied from 1–4 weeks, depending on the staff person’s availability. At follow-up visits, if the provider found that expected results were not in the EHR, then medical assistants helped to retrieve the faxed results from the data-entry queue.
Bidirectional Lab Interface Intervention
As part of a larger national demonstration project [17, 18], CARE received funds to enhance the clinic’s data management system by establishing a bidirectional lab interface. After the intervention, lab ordering didn’t change for the physician but the orders were transmitted directly to LabCorp. Phlebotomists then used a computer printout to guide specimen collection rather than entering the orders manually. The policy of obtaining tests 2–3 weeks in advance of future visits, when appropriate, did not change.
Test results were returned electronically into the EHR without needing to be data-entered. However, physicians continued to use faxed results to screen for abnormalities, and they used marked-up fax pages to task staff with scheduling follow-up visits, in the same manner as before the intervention. This manual process was retained because the EHR’s interface for reviewing lab results required substantial scrolling and paging to see all of a given patient’s results, did not highlight abnormalities, did not show reference ranges, and lacked a function to task staff with scheduling follow-up for abnormalities. Physicians perceived that the lab interface required about 1 minute per patient vs. less than 10 seconds per patient for reviewing and routing the printed results. Thus, after the intervention, the primary changes in work processes were the elimination of the results data entry and the elimination of results pursuit from the data-entry queue when patients returned for follow-up visits. The staff time saved went primarily to enhanced patient telephone contact.
EHR Data Extraction
We created a patient cohort consisting of the 1181 patients who had any encounter during the year prior to implementation of the bidirectional lab interface (12/1/2007 to 11/30/2008). Encounters included (1) face-to-face visits with a physician, nurse practitioner, registered nurse, social worker, dietician or adherence counselor, (2) injections, (3) TB checks, (4) lab draws, (5) telephone calls that resulted in an EHR note, (6) medication refills, (7) chart updates (lab, diagnostic results, etc) or (8) sub-specialty referrals. For this cohort, information systems personnel at the clinic then extracted and de-identified data from the NextGen EHR covering the time period from 12/1/2007 through 11/30/2010. The data extracted included patient demographics, insurance and other characteristics collected for purposes of regulatory reporting (income, homeless status), provider visit dates, laboratory test dates and values, and prescriptions and prescription dates. In the data used for all analyses, patients were identified only by a study ID that was assigned during the de-identification process and all patient-identifying fields were deleted.
Survey Methods
We conducted anonymous cross-sectional interviews in November 2008 and February 2011 with consecutive patients recruited from the clinic waiting room. Patients were not approached for recruitment; rather a sign posted at the registration desk announced the interview opportunity and patients had to request participation. Interviews were conducted using computer-assisted personal interviewing software (CAPI) except for questions regarding sensitive content areas (sexual behaviors, drug use), for which audio computer-assisted self-interview (ACASI) modules were used. The interview instrument was designed based on a review of the literature and consultation with experts involved with the Consumer Assessment of Healthcare Providers and Systems survey (CAHPS) [19]. The final instrument included the CAHPS 4-item general communication scale (how often providers listen carefully, explain things in an understandable way, show respect for what respondents had to say, and spend enough time with respondents), two overall satisfaction items from CAHPS (rating overall quality of care, recommendation of place to others), and four provider trust items taken from two sources [20, 21] (trust HIV care team to offer high quality care, trust HIV care team to put your medical needs above all other considerations, doctor will do whatever it takes to get me all the care I need, doctor only thinks about what is best for me). Based on the wording of CAHPS communication items, we also created 4 new items assessing the quality of communication specifically about lab tests, including items that assessed how often providers explained the tests ordered in a way that was easy to understand, how often providers explained the results of tests in a way that was easy to understand, how often providers explained the results of viral load tests in a way that was easy to understand, and how often providers explained the results of CD4 count tests in a way that was easy to understand.
Analysis
We constructed variables representing intensifications of the patient’s ART regimen (including the initiation of ART) and changes in laboratory values that could constitute a “signal” for changes in disease control that might warrant initiation or adjustment of ART [12]: increases in VL by a half-log (relative) or from below to above 100,000 copies/ml (absolute value), or decreases in CD4 count from above to below 350 or from above to below 200. These signals were classified as having a regimen-change response if they had a new ART medication (at the ingredient level) prescribed within 1–80 days after the signal. The 80-day threshold was selected by reviewing with the clinic’s lead physician cases of ART regimen changes that took place within 61 to 90 days of a signal. For all cases having a delay time less than 80 days, the regimen change appeared to be a response to the signal, whereas regimen changes that took place more than 80 days after the signal appeared to be more likely due to other factors such as side effects or subsequent laboratory changes.
To assess whether good disease control was obtained after medication changes made in response to signals, we constructed a variable representing whether the patient had achieved an undetectable VL result within the observation period and before any subsequent medication change.
We used Chi-square or Fischer exact tests to assess bivariate associations among categorical variables and t-tests or ANOVA tests to compare the values of continuous variables between groups. To explore associations of patient characteristics with signal-response times we used multivariate linear regression. Statistical calculations were carried out in SAS version 9.2 (SAS Institute, Cary, NC). All signal-response time and viral load analyses were adjusted for clustering of observations within patient using Taylor series variance estimation in the SAS SURVEYMEANS and SURVEYREG procedures. All reported P values are 2-tailed. The study was approved by the institutional review boards both at UCLA and St. Mary’s Medical Center.
Time from laboratory signal to change in therapy
Over the 3-year observation period, 547 patients (46% of the 1181-patient cohort) had a total of 1093 sets of laboratory results that qualified as a “signal” possibly warranting a change in therapy. Among these signal events, there were 171 (among 144 patients) that were followed by a change in the patient’s ART regimen within 80 days. Anecdotally, the majority of laboratory signal events were due to identifiable interruptions in medication adherence, for example associated with substance use relapses. Efforts for these patients focused on adherence rather than medication changes. Other factors accounting for signals with no response included patients known to have multidrug resistant HIV, patients declining regimen changes, and changes that constituted clinically insignificant fluctuations of CD4 or VL values around the usual treatment thresholds. As shown in Table I, the subset of patients having a signal-response pair tended to have less private insurance and greater use of ambulatory visits, adherence counseling, HAART therapy, case management visits, and mental health services than those who had no signal-response pair, but they were otherwise similar in all demographic characteristics.
Table 1
Table 1
Baseline Characteristics of All Patients vs. Those with Both a Signal and Response
Table II shows details for each type of signal, including its frequency, frequency of response, and response examples. Of note, many signal events were co-occurrences of more than one type of change.
Table II
Table II
Occurrence of each Laboratory Signal Type and Signal-Response Case Examples
For the 171 signal-response pairs, the median time from signal to response (the prescription of a new ART medication) was 34 days (range 1–78, inter-quartile range 18–54, mean 36.1). After adjustment for clustering by patient, the overall mean response time was 35.9 days. Figure 1 charts the adjusted mean response time by quarter over the 3-year observation period. Of note, the response time jumped from an adjusted mean of 37.7 days during the pre-intervention year to 48.2 days during the quarter immediately following activation of the lab intervention (an increase of 10.5 days, 28%). It then declined to a mean of 31.4 days over the remaining post-intervention period (a decrease of 6.3 days, 17% from the pre-period, P=0.03). No patient characteristics were significantly associated with the response time.
Figure 1
Figure 1
Mean Signal-Response Time (Adjusted for Clustering) by Quarter, Pre-and Post- Lab Intervention
Process changes associated with the lab interface intervention
Overall, patients had an average of 0.97 follow-up provider visits after the signal date and before the response prescription date. This average was 1.15 before the intervention, 1.19 during the implementation quarter, and 0.73 for the remaining post-intervention period (a 37% decrease vs. the pre- period, P=0.10). Forty four percent of signal events occurred on a date with no provider visit, implying that the patient came in for testing prior to a visit, per the clinic’s desired protocol. The average response time for these cases was 33.5 days vs. 37.8 days for others (P=0.19 for the 11% difference). By time period, this pre-visit testing rate was 40% before the intervention, 31% during the first post-implementation quarter, and then 51% for the remaining period (P=0.28). We also found that response prescriptions occurred on dates with no visit in 31% of cases, demonstrating that prescriptions were sometimes called in (not the clinic’s desired protocol); this proportion was 26% before, 38 % during, and 35% after (P=0.49). Finally, the average time between follow-up events (follow-up visits between signal and response plus the response itself, whether or not it occurred at a visit) increased from 21 to 29 days during the first quarter after the intervention, and then it returned to 21 days over the remaining observation period (a 38% increase during the first post-implementation quarter, P=0.07).
Achievement of undetectable viral load
For 23 of the 171 signal-response pairs (13%), the signal involved a change in CD4 count only along with a VL that remained undetectable; these were excluded as not informative for the analysis of achieving undetectable VL. Another 11 signal-response pairs (6.4%) were excluded from this analysis because there were no VL lab results during the observation period subsequent to the ART regimen change. Among the remaining 137 medication-change events that were made when the patient had a detectable VL, 89 (65%) were followed by achievement of an undetectable VL during the observation period and before any subsequent medication change, whereas 48 (35%) never achieved an undetectable VL during this time. As shown in Figure 2, the probability of achieving an undetectable VL was not associated with the signal-response time. The mean signal-response delay was 35.5 days for medication changes that were followed by the patient achieving undetectable status vs. 36.3 days for changes that were followed by only detectable viral loads.
Figure 2
Figure 2
Achievement of Undetectable Viral Load (VL) after an Antiretroviral Medication Change, by Signal-Response Time
Survey
Table III compares the cross-sectional patient survey responses before and after implementation of the lab interface. There were statistically significant differences in respondent race, age, and HIV risk factor, but not in other characteristics. The differences were relatively small, for example a 3-year difference in mean age. After implementation of the laboratory interface, patients rated communication about laboratory tests significantly higher, on 3 of 4 individual items and when these items were combined into a scale score. By comparison, patients did not rate general communication or trust in their doctors and HIV team more highly. There was a trend toward higher ratings on the overall satisfaction scale after the lab interface but these did not achieve the conventional level of statistical significance for the scale or for either of the component questions individually.
Table III
Table III
Patient Survey Respondent Characteristics and Perceptions of HIV Care
Implementation of the bi-directional lab interface was associated with a modest but statistically significant improvement in the time required for the clinic to enact regimen changes after important HIV-specific changes in laboratory test results. This response time represents time during which the HIV disease is not in control. The improvement of one week in this delay could be clinically important—for reducing the potential for forward transmission of HIV in addition to reducing the direct health effects of HIV and the risk of opportunistic infection for the individual patient. Because delays in therapy could increase the potential for mutations to result in resistant strains, we had hypothesized that longer signal-response times might decrease the probability of achieving an undetectable viral load. One prior study found that delay in ART initiation was associated with failure to recover normal CD4 cell counts[22]. Our results did not support this hypothesis, but the incremental delays in our study were shorter than those documented in the prior study.
The improvement we observed in response times was mediated by a decrease in the number of follow-up visits between signal and response. In the present study, implementation of the lab interface resulted in only a few direct process changes that could have affected the clinic’s response times. First, phlebotomists no longer transcribed orders, eliminating a potential source of error in the tests obtained. Such errors could have increased response-time if there were errors in follow-up testing after the initial signal (to recheck control after an adherence intervention or to check viral genotype for resistance). No data was available on the frequency of errors in test-ordering, but occasional errors in ordering genotype or other follow-up tests after a signal could have led to some additional follow-up visits. Probably a greater source of reduction in follow-up visits, however, was achieved through eliminating the queue of un-entered results. Test results were occasionally unavailable in the EHR when patients returned for follow-up, creating disruption for physicians and for staff who were tasked with finding the printed results in such cases. Unavailable results probably led to some additional visits. In addition, the staff time freed up from pursuing results may have enabled better coordination for obtaining lab tests in advance of future clinic visits. Although the 20% improvement we observed in the use of pre-visit testing wasn’t statistically significant, these incidents were associated with slightly shorter response times. The increased availability of results in the EHR probably also explains the improvement that patients noticed in communication about laboratory tests.
The fact that clinicians continued to screen incoming results on fax pages, in essence working around the EHR's features for this, indicated a need for better EHR usability and matching of EHR design with clinical workflows. Recent efforts to decompose the tasks of laboratory result management represent an important step toward the broader goal of improving EHR support for these tasks [3, 23] These efforts will need to be paired with ongoing evaluation and monitoring of delays in test result follow-up [24]. Studies such as the present one, evaluating incremental changes in EHR systems, will also be increasingly important as EHR adoption spreads.
Finally, we observed a common pattern of change in performance after implementation of a system change—an initial dip in performance followed by improved performance over subsequent time periods. Although this phenomenon is familiar to implementers, the health information technology (IT) literature contains more reports of outright implementation failures (such as one detailed report of implementation failure for a laboratory order-entry system [25]) than reports of temporary performance setbacks due to EHR adoption. We are aware of only one article documenting this phenomenon around EHR implementation [26], and we were unable to find any additional reports in searching Pubmed. Furthermore, we were able to find only a few studies in the broader implementation literature that deal formally with this performance-dip phenomenon (e.g., [27]). More commonly, studies evaluating the impact of health IT interventions deal with this phenomenon by simply excluding the initial months or even a year after the implementation. For example, a study of response times to abnormal potassium results before and after EHR implementation excluded an entire year following EHR implementation [28]. More research is needed to understand and plan for minimizing these performance setbacks, which can be substantial, as we experienced. Although we do not have direct evidence for the mechanism underlying this effect, we did find that the time between follow-up events was 39% longer in the post-implementation quarter than in the pre-intervention or remaining post-intervention periods. This additional delay was probably due to increased workloads for scheduling and telephone staff during the transition time when some lab results were available immediately in the EHR system while older results (and a few results from non-connected labs) still remained in the data-entry queue, which was still being worked down by the same staff. These results further suggest that dips in follow-up performance might be remediable through hiring temporary staff or offering over-time during the vulnerable period to staff responsible for patient scheduling and telephone follow-up.
In conclusion, adding electronic laboratory data exchange in HIV care holds promise for improving the timeliness of important therapy changes and for improving patient-centered communication about laboratory tests. However, more extensive process re-engineering, supported by more substantial redesign of EHR user interfaces, may enable greater benefits to be realized from laboratory data exchange. Provider organizations should also plan to compensate for dips in follow-up performance during the initial months after implementation of new HIT capabilities.
Research Highlights
  • Electronic interchange of test orders and results between EHRs and clinical laboratories is expected to improve care by ensuring that the correct tests are obtained and that results are matched and followed up in a timely manner for all ordered tests.
  • Few studies have examined the effects of implementing electronic laboratory data exchange within electronic health records.
  • Our study, the first we are aware of to examine electronic laboratory data exchange in HIV care, found that it improved the timeliness of adjusting antiretroviral medications and patients’ assessments of communication about laboratory tests.
  • Further research is needed on redesigning workflows and user interfaces to take greater advantage of the data that is made available through electronic data exchange.
Summary Table
What was already known on this topicWhat this study added to our knowledge
  • Electronic interchange of test orders and results between EHRs and clinical laboratories may improve care by ensuring that the correct tests are obtained and that results are matched and followed up in a timely manner for all ordered tests.
  • Few studies have examined the effects of implementing electronic laboratory data exchange within electronic health records.
  • Timely responses to laboratory results are critical in HIV care, both for the patient and for reducing the potential for viral transmission.
  • Electronic laboratory data exchange could significantly improve the timeliness of adjusting antiretroviral medications for HIV patients.
  • It also holds promise for improving patient-centered communication about laboratory tests.
  • Further research is needed on redesigning workflows and user interfaces to take greater advantage of the data available.
Acknowledgements
This research was supported by a grant from the Health Resource and Services Administration (HRSA-07-046). Dr. Cunningham received partial support for his time on this study from the NIDA (R01 DA030781), and the NIMH (R34 MH089719). Drs. Bell and Cunningham Mr. Nakazono had partial support from the National Center for Advancing Translational Science (UL1TR000124). The funders had no role in the analysis and interpretation of data or in the writing of the manuscript. We are grateful to Jimmy Ngo for assistance in preparing the manuscript.
Footnotes
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Authors’ contributions
Each of the authors (Bell, Cima, Seiden, Nakazono, Alcouloumre, and Cunningham) contributed to the conception and design of the study, to interpretation of data, and to drafting or critically revising the article and to final approval of the version submitted.
Conflict of interest statement
None of the authors has any financial or personal relationships with people or organizations that could inappropriately influence bias or bias the work reported in this paper.
1. Elder NC, McEwen TR, Flach J, Gallimore J, Pallerla H. The management of test results in primary care: does an electronic medical record make a difference? Fam Med. 2010 May;42(5):327–333. [PubMed]
2. Elder NC, McEwen TR, Flach JM, Gallimore JJ. Management of test results in family medicine offices. Ann Fam Med. 2009 Jul-Aug;7(4):343–351. [PubMed]
3. Yackel TR, Embi PJ. Unintended errors with EHR-based result management: a case series. J Am Med Inform Assoc. 2010 Jan-Feb;17(1):104–107. [PMC free article] [PubMed]
4. Murff HJ, Gandhi TK, Karson AK, Mort EA, Poon EG, Wang SJ, et al. Primary care physician attitudes concerning follow-up of abnormal test results and ambulatory decision support systems. Int J Med Inform. 2003 Sep;71(2-3):137–149. [PubMed]
5. Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson AS, Bates DW. "I wish I had seen this test result earlier!": Dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004 Nov 8;164(20):2223–2228. [PubMed]
6. Sujansky WV, Overhage JM, Chang S, Frohlich J, Faus SA. The development of a highly constrained health level 7 implementation guide to facilitate electronic laboratory reporting to ambulatory electronic health record systems. J Am Med Inform Assoc. 2009 May-Jun;16(3):285–290. [PMC free article] [PubMed]
7. Dixon BE, Zafar A, Overhage JM. A Framework for evaluating the costs, effort, and value of nationwide health information exchange. J Am Med Inform Assoc. 2010 May-Jun;17(3):295–301. [PMC free article] [PubMed]
8. Vest JR, Jasperson J. What should we measure? Conceptualizing usage in health information exchange. J Am Med Inform Assoc. 2010 May-Jun;17(3):302–307. [PMC free article] [PubMed]
9. Frisse ME, Johnson KB, Nian H, Davison CL, Gadd CS, Unertl KM, et al. The financial impact of health information exchange on emergency department care. J Am Med Inform Assoc. 2011 Nov 4; [PMC free article] [PubMed]
10. Herwehe J, Wilbright W, Abrams A, Bergson S, Foxhood J, Kaiser M, et al. Implementation of an innovative, integrated electronic medical record (EMR) and public health information exchange for HIV/AIDS. J Am Med Inform Assoc. 2011 Oct 28; [PMC free article] [PubMed]
11. Johansen I, Rasmussen M. Electronic interchange of lab test orders and results between laboratories reduces errors and gives full traceability. Stud Health Technol Inform. 2010;155:65–68. [PubMed]
12. Thompson MA, Aberg JA, Cahn P, Montaner JS, Rizzardini G, Telenti A, et al. Antiretroviral treatment of adult HIV infection: 2010 recommendations of the International AIDS Society-USA panel. JAMA. 2010 Jul 21;304(3):321–333. [PubMed]
13. Attia S, Egger M, Muller M, Zwahlen M, Low N. Sexual transmission of HIV according to viral load and antiretroviral therapy: systematic review and meta-analysis. AIDS. 2009 Jul 17;23(11):1397–1404. [PubMed]
14. Donnell D, Baeten JM, Kiarie J, Thomas KK, Stevens W, Cohen CR, et al. Heterosexual HIV-1 transmission after initiation of antiretroviral therapy: a prospective cohort analysis. Lancet. 2010 Jun 12;375(9731):2092–2098. [PMC free article] [PubMed]
15. Fisher M, Pao D, Brown AE, Sudarshi D, Gill ON, Cane P, et al. Determinants of HIV-1 transmission in men who have sex with men: a combined clinical, epidemiological and phylogenetic approach. AIDS. 2010 Jul 17;24(11):1739–1747. [PubMed]
16. Charlebois ED, Das M, Porco TC, Havlir DV. The effect of expanded antiretroviral treatment strategies on the HIV epidemic among men who have sex with men in San Francisco. Clin Infect Dis. 2011 Apr 15;52(8):1046–1049. [PMC free article] [PubMed]
17. Magnus M, Herwehe J, Proescholdbell RJ, Lombard F, Cajina A, Dastur Z, et al. Guidelines for effective integration of information technology in the care of HIV-infected populations. J Public Health Manag Pract. 2007 Jan-Feb;13(1):39–48. [PubMed]
18. U.S. Department of Health and Human Services HRaSA . Enhancement & Evaluation of Existing Health Information Electronic Network Systems for PLWHA in Underserved Communities. 2012 [26 Jan 2012]; Available from: http://hab.hrsa.gov/abouthab/special/underservedcommunities.html.
19. U.S. Department of Health and Human Services AfHRaQ. Consumer Assessment of Healthcare Providers and Health Systems Survey. 2012 [22 Jan 2010]; Available from: http://www.cahps.ahrq.gov/
20. Hall MA, Zheng B, Dugan E, Camacho F, Kidd KE, Mishra A, et al. Measuring patients' trust in their primary care providers. Med Care Res Rev. 2002 Sep;59(3):293–318. [PubMed]
21. Rodriguez HP, Wilson IB, Landon BE, Marsden PV, Cleary PD. Voluntary physician switching by human immunodeficiency virus-infected individuals: a national study of patient, physician, and organizational factors. Med Care. 2007 Mar;45(3):189–198. [PubMed]
22. Kelley CF, Kitchen CM, Hunt PW, Rodriguez B, Hecht FM, Kitahata M, et al. Incomplete peripheral CD4+ cell count restoration in HIV-infected patients receiving long-term antiretroviral treatment. Clin Infect Dis. 2009 Mar 15;48(6):787–794. [PMC free article] [PubMed]
23. Tarkan S, Plaisant C, Shneiderman B, Hettinger AZ. Reducing missed laboratory results: defining temporal responsibility, generating user interfaces for test process tracking, and retrospective analyses to identify problems. AMIA Annu Symp Proc. 2011;2011:1382–1391. [PMC free article] [PubMed]
24. Singh H, Thomas EJ, Sittig DF, Wilson L, Espadas D, Khan MM, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med. 2010 Mar;123(3):238–244. [PMC free article] [PubMed]
25. Peute LW, Aarts J, Bakker PJ, Jaspers MW. Anatomy of a failure: a sociotechnical evaluation of a laboratory physician order entry system implementation. Int J Med Inform. 2010 Apr;79(4):e58–e70. [PubMed]
26. Baron RJ, Fabens EL, Schiffman M, Wolf E. Electronic health records: just around the corner? Or over the cliff? Ann Intern Med. 2005 Aug 2;143(3):222–226. [PubMed]
27. Edmondson AC, Bohmer RM, Pisano GP. Disrupted Routines: Team Learning and New Technology Implementation in Hospitals. Administrative Science Quarterly. 2001;46(4):685–716.
28. Lin JJ, Moore C. Impact of an electronic health record on follow-up time for markedly elevated serum potassium results. Am J Med Qual. 2011 Jul-Aug;26(4):308–314. [PubMed]