|Home | About | Journals | Submit | Contact Us | Français|
To evaluate practitioners' expectations of, and satisfaction with, older and newer electronic health records (EHRs) after a transition.
Pre- and post-transition survey administered at six academic-affiliated ambulatory care practices from 2006 to 2008. Four practices transitioned to one commercial EHR and two practices to another. We compared respondents' expectations of, and satisfaction with, the newer EHR.
523 subjects were eligible: 217 were available before transition and 306 after transition. 162 pre-transition and 197 post-transition responses were received, yielding 75% and 64% response rates, respectively. Practitioners were more satisfied with the newer EHRs (64%) compared with the older (56%) (p=0.15) and a small majority (58%) were satisfied with the transition. Practitioners' satisfaction with the older EHRs for completing clinical tasks was high. The newer EHRs exceeded practitioner expectations regarding remote access (61% vs 74%; p=0.03). However, the newer EHRs did not meet practitioners' expectations regarding their ability to perform clinical tasks, or more globally, improve medication safety (81% vs 61%; p<0.001), efficiency (70% vs 44%; p<0.001), and quality of care (77% vs 67%; p=0.04).
Most practitioners had favorable opinions about EHRs and reported overall improved satisfaction with the newer EHRs. However, practitioners' high expectations of the newer EHRs were often unmet regarding facilitation of specific clinical tasks or for improving quality, safety, and efficiency.
To ensure practitioners' expectations, for instance regarding improvements in medication safety, are met, vendors should develop and implement refinements in their software as practices upgrade to newer, certified EHRs.
Adoption of ambulatory electronic health records (EHRs) has become important for improving patient safety, healthcare quality, and efficiency.1–3 Nationally, up to $30 billion will be spent on the EHR incentive program to support meaningful use of EHRs.4 EHRs certified for meaningful use have the capability to record patients' clinical and demographic data, view and manage laboratory test results and imaging studies, manage order entry, and support clinical care through clinical decision support.5 Early adopters of EHRs are progressively transitioning to newer EHRs that meet meaningful use requirements in order to be eligible for financial incentives.6 It is important to identify and describe the barriers faced while transitioning to newer EHRs, since it is well known that the initial transition from a paper-based to an electronic system can be challenging and transitioning between systems may have different barriers.2 7–9
Limited research evaluating transitions between EHR systems suggests that a number of unique challenges may be faced when transitioning from an older to a newer system.9 These challenges include resistance from practitioners who may be very loyal to an older EHR system, issues surrounding data transfer between two different systems, and potential threats to patient safety due to variations in the e-prescribing functionality.10 This is in addition to traditional challenges faced by practitioners newly implementing an EHR, such as the intensive training and ongoing technical support needs and perceived disruptions to workflow and efficiency that come from system implementation.6 11–14
As key healthcare policy changes begin to address and overcome some of these barriers by providing financial and technical support for practitioners and organizations adopting EHRs, overcoming ambulatory practitioners' apprehension regarding EHRs is becoming ever more important.2 15 16 A recent review of health information technology (health IT) literature found that although the benefits of health IT are beginning to emerge in both smaller and larger organizations, provider dissatisfaction with EHRs remains a significant problem and thus has the potential to be an important barrier to widespread adoption and meaningful use of EHRs.17 Understanding practitioners' expectations of a new EHR system before the transition and their satisfaction afterwards may help to improve physician satisfaction following the changeover.
Therefore, we carried out a pre- and post-transition study designed to evaluate practitioners' expectations of and satisfaction with transitioning from older EHRs to newer EHRs at six academic ambulatory care practices. Understanding practitioner perceptions can facilitate the development of strategies to maximize satisfaction while minimizing resistance surrounding transitions between EHR systems.
We interviewed practitioners at six ambulatory care practices located in New York City. Specialties included internal medicine, obstetrics/gynecology, pediatrics, geriatrics, and family medicine. These six ambulatory care practices are part of the ambulatory care network of a large teaching hospital associated with two medical schools. The network is distributed between two geographic locations; four practices are located at an eastern campus and two at a western campus. Patients in these practices receive ongoing care provided by residents, under the supervision of faculty members, or care provided directly by attending physicians or nurse practitioners. Although resident education is a significant activity at these locations, the majority of attending practitioners have their own patient panels. Furthermore, the few attending practitioners who solely supervise residents not only have to co-sign the resident's note, but also are held responsible for ensuring patient charts are completely up-to-date (eg, visit notes, problem lists, medication lists, and allergy information). Therefore, all practitioners were required to know how to use the newer EHRs.
In 2005, the ambulatory care network began planning for the implementation of new EHRs for all practice sites. Two different commercial EHRs were implemented, with all the practices on the eastern campus adopting one commercial system and all practices on the western campus adopting a different commercial system. Prior to the implementation of the new EHRs, the electronic capabilities of each office practice varied. The East campus practices used an older EHR which did not allow practitioners to record extensive patient encounters, review specialized laboratory tests (eg, pathology specimens or radiology results), or write prescriptions, or provide remote access. The West campus practices used a hybrid electronic and paper system to complete tasks such as scheduling patients, ordering and reviewing laboratory tests, and writing prescriptions. For example, practitioners had the option of creating prescriptions either by hand or electronically. History and physical examination results were documented using paper, and remote access was unavailable.
Preparation for the transition was extensive. The old EHRs had been in place for at least 5 years at all sites. Prior to the implementation of the newer EHRs, all practitioners had to attend mandatory training sessions with separate sessions for faculty and residents. Additionally, each practice site had designated physician ‘super users’ who had extra training to assist with the on-site training of practitioners after the system went live. For the first month after the system went live, practitioners' schedules were reduced and support staff from each vender were onsite at practices implementing their software. After the first month, the vendor support team was still available via email, and monthly meetings were held between the vendor and representatives from each practice site.
This was a pre–post, cross-sectional survey of ambulatory care practitioners. All practitioners affiliated with the six ambulatory care practices were eligible for the study. Practitioners on maternity or disability leave were excluded. Additionally, for the pre-transition survey, we excluded residents in their last year of training since they would graduate before the post-transition survey was fielded. Practitioners varied by specialty and training level. This study received human subject research approval from the Institutional Review Boards at Weill Cornell Medical College and the College of Physicians and Surgeons of Columbia University.
We developed an eight-page survey tool based on a systematic review of the literature regarding practitioner satisfaction with, and adoption of, EHRs and computerized physician order entry (CPOE) systems, focusing specifically on ambulatory care practices. The survey was piloted with eight practitioners of various backgrounds, training, and practice locations.
The final pre-transition survey included items designed to assess practitioner and practice characteristics, general work perceptions, and comfort with and use of IT. Specifically, practitioners were asked to respond to questions regarding clinical tasks, their method of completing these tasks (on paper or electronically), and satisfaction with their method of completing these tasks. Respondents were also asked questions regarding expected satisfaction, benefits, barriers, and the costs of implementing a new EHR. The post-transition survey differed slightly from the pre-transition survey. Two new questions were added (‘How satisfied are you with this new EHR?’ and ‘How satisfied were you with the transition to the new EHR?’) and questions regarding anticipated satisfaction with the new EHR were eliminated as they were no longer relevant. The post-transition survey was piloted with six practitioners of various backgrounds, training, and practice locations.
For the pre-transition survey, Atlantic Research and Consulting (Boston, Massachusetts, USA) was contracted to administer the survey from March to June 2006 over a 10-week period on each campus. The initial survey was mailed with a $10 cash incentive and an introductory letter which included a link for the web version of the survey. Subsequently, a second mailing and two reminder emails with web links to the questionnaire were sent to non-respondents.
Approximately 3 months after the newer EHRs were implemented at each practice site, we administered the post-transition survey. The survey was administered both electronically and via mail, which included a link for the web version of the survey, along with a $5 Starbucks gift card incentive. Additionally, we sent weekly reminder emails to non-respondents and visited each practice twice to promote survey completion. The post-transition survey was administered from February to November 2008, with each practice site given a 12-week period to complete the survey.
We compared respondents' expectations of, and eventual satisfaction with, the new EHR and its implementation across three domains: practitioner characteristics, workflow satisfaction, and comfort with and use of IT. Practitioner characteristics included: age, gender, specialty training, training level (staff physician or trainee), practice site, years practicing medicine, and years at the practice site. Practitioner workflow satisfaction included: satisfaction with the old system (satisfied or dissatisfied), satisfaction with the newer EHR (satisfied or dissatisfied), attitude toward EHRs in general (responses grouped as very or somewhat positive compared to somewhat or very negative), and satisfaction with the transition to the newer EHR (satisfied or dissatisfied). Comfort with and uses of IT included: self-reported typing skills (excellent, good, poor, and very poor) and perceived comfort level with using IT to perform clinical duties (very comfortable or not very comfortable).
All analyses were conducted using SAS Statistical Package V.9.1 (SAS Institute). Comparisons between pre- and post-transition survey respondents were made using Pearson χ2 analyses. Then we described and compared responses about work flow, comfort with IT, expectations, and satisfaction, before and after the transition using Pearson χ2 analyses or Fisher's exact tests when appropriate. Lastly, as we excluded respondents who indicated that they did not perform a clinical activity, our denominators for each item vary accordingly.
We performed two sub-analyses to check the validity of our results. First, we examined a subgroup of sites consisting of the four practices that transitioned from the same older EHR to the same newer EHR. Pre- and post-transition comparisons were made for this subset using the same variables as described for the entire dataset. These associations were examined using the χ2 and Fisher's exact test. The second sub-analysis was for a group of 45 subjects who responded to both the pre- and post-transition surveys to examine within-subject differences. McNemar's test for paired data was used to identify differences between pre- and post-transition responses. In both sub-analyses, the results showed similar trends as in the full dataset. We show our main analysis and the sub-analysis of the 45 practitioners who responded to both the pre- and post-transition surveys. However, sample size constraints did not allow for statistical comparisons between the two analyses.
In total, 523 subjects were eligible to participate in the survey, 217 before transition and 306 after transition. Completed surveys were received from 162 pre- and 197 post-transition respondents, yielding a 75% and 64% response rate, respectively. Pre- and post-transition respondents were similar with respect to gender, age, profession, training level, number of years since medical school graduation, and number of years at practice site (table 1). Respondents differed in specialty, with more geriatric and family medicine physicians responding to the post-transition survey.
Response rates at each practice site before and after transition varied (table 2). Only the geriatric practice had an increased response rate. The other practices had decreased or similar post-transition survey response rates.
Overall, the majority of practitioners (90%, 177/196) reported a very or somewhat positive attitude toward EHRs in general. Practitioners' satisfaction increased with the newer EHRs: 56% (91/162) were very or somewhat satisfied with the older system compared to 64% (123/193) with the new EHR (p=0.15). A small majority (58%, 110/191) of practitioners reported that they were very or somewhat satisfied with the transition to the new EHR.
Practitioners in this study were comfortable with using IT in general and with using it for patient care (93% (149/161) before transition compared to 97% (188/194) after transition; p=0.06). The majority of respondents also reported having excellent or good typing skills (88% (141/161) before transition, 92% (180/195) after transition; p=0.24).
Practitioners had significantly higher satisfaction rates with the old EHRs in supporting their ability to perform most clinical tasks (table 3). For example, practitioners' satisfaction with documenting patient histories significantly decreased after transition (90% vs 73%; p<0.001). Similarly, practitioners were less satisfied with documenting allergies, keeping patient lists, tracking health maintenance systems, referring to clinical guidelines, checking drug information, and writing prescriptions. Increased satisfaction was reported with communicating referrals and having remote access. Lastly, there was no change in practitioners' satisfaction with ordering or reviewing laboratory tests and results, coding according to ICD-9, or creating patient registries.
Practitioners had high expectations regarding the new EHRs' capabilities for aiding the performance of particular tasks. Actual satisfaction after transition exceeded pre-transition expected satisfaction for only two tasks—remote access and viewing laboratory results (p=0.03 and 0.04, respectively) (table 3). Post-transition satisfaction was lower than expected for three tasks—keeping lists, tracking health maintenance, and availability of clinical guidelines. There was no difference in pre-transition expected satisfaction and actual post-transition satisfaction for the remainder of the clinical tasks.
When more global pre–post expectations, such as expectations related to the impact of the new EHR on safety, efficiency, and quality of care, were evaluated, pre-transition expectations were generally significantly higher compared with post-transition satisfactions (table 4). For example, 81% of practitioners surveyed before transition expected that the new EHR would improve medication safety; however, only 61% maintained this satisfaction after transition (p<0.001). The same was true regarding expectations that the new EHR would improve efficiency and productivity for practitioners and office staff, overall work life, workload, clinical decision-making, and quality of care. There were no areas in which global satisfactions after transition significantly exceeded expectations before transition.
A total of 45 respondents completed both the pre-transition and post-transition surveys. In contrast to the overall sample, more staff physicians (67%, 26/39) than trainees (33%, 13/39) responded. These respondents were older (mean age 41 years), were more often female (64%, 29/45), and from fewer specialties, with 47% (17/34) in pediatrics, 33% (12/34) in internal medicine, and 14% (5/34) in obstetrics and gynecology.
Overall, the within-group analysis showed similar trends as described earlier with the total sample except fewer items were statistically significant. For example, when comparing post-transition satisfaction with pre-transition expected satisfaction, the overall analysis found three tasks (ie, keeping lists, tracking health maintenance, and availability of clinical guidelines) for which pre-transition expectations were higher than post-transition satisfaction, whereas the within-group analysis found only one of these clinical tasks to be statistically significant, with the other two showing similar trends. Specifically, the within-group analysis showed the decline in satisfaction with tracking health maintenance was statistically significant (86% (19/22) before transition compared to 50% (11/22) after transition; p=0.02) and a trend for decreasing satisfaction with respect to keeping lists (72% (23/32) vs 30% (19/32); p=0.16) and availability of clinical guidelines (56% (9/16) vs 44% (7/16); p=0.32). In the same manner, analysis of the total sample found two tasks for which post-transition satisfaction exceeded pre-transition expected satisfaction (ie, remote access and viewing laboratory results), whereas the within-group analysis found a similar non-statistical trend for remote access (69% (11/16) vs 88% (14/16); p=0.08) but no change in viewing laboratory results (77% (27/35) vs 74% (26/35)). There were no other statistical differences in pre-transition expected satisfaction and actual post-transition satisfaction for the remainder of the clinical tasks.
When evaluating more global pre–post expectations, such as expectations related to the impact of the new EHR on safety and efficiency, this pattern was also found, with the within-group analysis mirroring the results of the overall sample, but with fewer items meeting statistical significance. For example, similarly to the overall results, the within-group analysis found that pre-transition expectations were generally higher compared with post-transition satisfaction. For instance, 78% (31/40) of the within-group practitioners before transition expected that the new EHR would improve medication safety, but 60% (24/40) of these practitioners had this same expectation after transition (p<0.03). Likewise, this was true for efficiency and productivity for practitioners (66% (27/41) vs 41% (17/41); p=0.01) and office staff (63% (25/40) vs 40% (16/40); p=0.01), and for workloads (54% (22/41) vs 32% (13/41); p=0.02). The other expectations had similar trends but were not statistically significant (ie, work life (54% vs 39%; p=0.130), patients' privacy (41% vs 29%; p=0.09), and access to medical records (93% vs 85%; p=0.3)).
Among a sample of primary care practitioners transitioning from an older to a newer EHR with many more system capabilities, we found practitioners generally had positive attitudes overall toward EHRs and satisfaction was higher with the new EHR system. However, pre-transition expectations of the new EHRs were high. While some of the pre-transition expectations were met by the new EHR, particularly the ability to provide remote access, some were not, including the ability of the new EHR to improve medication safety, efficiency, and quality of care.
The sub-group analysis of practitioners who responded to both the pre-transition and post-transition surveys, supports the trends found in the overall analysis. Importantly, since the results of the sub-analysis, which consisted of a higher percentage of older, staff practitioners, mirrored the main analysis, we can reason that the perceptions of the main analysis were held by staff practitioners and trainees alike.
Our study is novel in being among the first to evaluate longitudinally practitioner expectations for, and then their satisfaction with, the transition between two EHR systems. This is important because meaningful use requirements may require practices to upgrade and transition from older to newer EHRs. Generally, although the majority of our respondents had positive attitudes toward EHRs after the transition, only two-thirds were very or somewhat satisfied with the new EHRs. Our lower satisfaction level compared to previous studies of practices adopting an EHR system (reporting rates around 80%–90%) suggests that transitioning between EHRs may present unique challenges.14 18–20 For example, practitioners who are very loyal to and satisfied with their old EHR may be more critical when comparing the functionalities of the older and newer systems. This is consistent with our results, which found that practitioners expressed lower levels of post-transition satisfaction with multiple task-specific functionalities of the new EHR.
Our findings that many practitioner expectations were unmet with regard to implementation of the new EHR system are mixed compared to the current literature.21–23 For example, DesRoches and colleagues found that among practitioners using a commercial electronic records system, many reported positive effects of the system on quality of clinical decisions, communication with other practitioners, prescription refills, and avoidance of medication errors.24 There may be several explanations why our findings differ. Previous literature has focused on respondents' initial adoption of an EHR, rather than on transitioning between EHRs. Moreover, although the new EHRs in our study had extensive functionalities, including in comparison to the functionalities available in the older EHRs, the presence of these functionalities alone may not lead to enhanced medication safety or quality of care, supporting practitioner perceptions.25 26 Lastly, we did not assess if practitioners were using the EHRs' functionalities correctly, although actual usage is an important requisite for achieving gains in quality, safety, and efficiency.6 25 27
Interestingly, despite lower levels of post-transition satisfaction with multiple functionalities of the new EHR system, practitioners were still more satisfied overall with the newer EHR. There is growing evidence suggesting that not all useful and important functions of EHRs are used or valued in a similar manner.6 24 25 27 28 It may be that practitioners were so satisfied with a few highly valued features of the new EHR system that it led to overall higher satisfaction with the new EHR system. In this case, practitioners' satisfaction with the ability to remotely access information was greatly increased after transition compared with before transition and may have contributed to the high overall satisfaction with the new EHR, especially since ability to remotely access information has important implications for workflow and the quality and safety of patient care. For example, with remote access, practitioners can utilize the EHR in multiple settings, including at home. This allows practitioners flexibility as to when and where to prepare for patient visits, complete notes, and review test results.
Identifying the EHR features most important to practitioners and optimizing their design in commercial systems may be critical to ensuring overall satisfaction among practitioners who transition between EHR systems. Managing practitioner expectations early in the process also appears to be crucial, since high expectations for a new EHR may not be met after transition. Lastly, given that only a small majority of practitioners in our study were very or somewhat satisfied with the transition between systems, identifying best practices for transitioning between systems will be important given the increasing number of practitioners or organizations expected to upgrade between system versions or transition between systems in order to be eligible for meaningful use incentives.
Our study had several limitations. First, although our study involved a large number of practitioners, from six separate practices that used two different systems, these practitioners were all associated with a large urban academic center, limiting generalizability. Second, due to our study design, we only performed between-group comparisons rather than within-subject comparisons. However, we were able to perform a small sub-analysis of practitioners who took both the pre- and post-transition surveys and found similar trends. Third, our study measured perceptions 3 months after implementation. It is possible that a longer period of acclimation to the new EHR would have affected our results. Lastly, differences in our response rates among practice sites and practitioner training levels may have introduced an unknown bias.
With approximately 24%–36% of practitioners using commercial EHRs and with a growing drive to have all practices adopt certified EHRs that will meet meaningful use requirements, this study contributes to the much needed literature about practitioner experiences in transitioning to a newer ambulatory EHR.24 29 30 Our research suggests that high expectations before transition may be unrealized for practitioners transitioning between EHR systems, and that this may impact practitioner satisfaction after transition. Understanding the unique perspective of practitioners transitioning between systems can allow for the development of better transition practices, improved pre-transition expectation management, and the design of EHRs that better reflect practitioner needs and priorities.
Funding: This project was supported by the United Hospital Fund (grant number 061118S) and New York-Presbyterian Hospital.
Competing interests: None.
Ethics approval: This study was conducted with the approval of Weill Cornell Medical College.
Provenance and peer review: Not commissioned; externally peer reviewed.