|Home | About | Journals | Submit | Contact Us | Français|
Increasing the use of electronic medical records (EMR) has been suggested as an important strategy for improving healthcare safety.
To sequentially measure, evaluate, and respond to safety culture and practice safety concerns following EMR implementation.
Safety culture was assessed using a validated tool (Safety Attitudes Questionnaire; SAQ), immediately following EMR implementation (T1) and at 1.5 (T2) and 2.5 (T3) years post-implementation. The SAQ was supplemented with a practice-specific assessment tool to identify safety needs and barriers.
A large medical group practice with a primary care core of 17–18 practices, staffed by clinicians in family medicine, pediatrics, internal medicine.
Survey results were used to define and respond to areas of need between assessments with system changes and educational programs.
Change in safety culture over time; perceived impact of EMR on practice.
Responses were received from 103 of 123 primary care providers in T1 (83.7 % response rate), 122 of 143 in T2 (85.3 %) and 142 of 181 in T3 (78.5 %). Safety culture improved over this period, with statistically significant improvement in all domains except for stress recognition. Time constraints, communications and patient adherence were perceived to be the most important safety issues. The majority of respondents in both T2 (77.9 %) and T3 (85.4 %) surveys agreed that the EMR improved their ability to provide care more safely.
Implementation of an EMR in a large primary care practice required redesign of many organizational processes, and was associated with improvements in safety culture. Most primary care providers agreed that the EMR allowed them to provide care more safely.
Despite efforts to improve patient safety, patients may still suffer harm during healthcare episodes. Because most healthcare services are delivered in primary care settings, understanding barriers to safe primary care is particularly important.1–3 Primary care physicians perceive time limitations and disorganized work systems as factors that influence safety,4,5 with communication, medication management, and test results tracking believed to be common sources of avoidable error.2,3,6,7 Work is underway to develop consensus about key processes and metrics to promote systematic safety monitoring and reduce risk in primary care.8
While working to develop these metrics, culture assessments have been used as indirect safety measures. Safety culture refers to a shared system of attitudes, values and practices that reflect an organization’s commitment to safety.9 An organization with a strong safety culture empowers staff to identify and report concerns, acknowledges errors, and allocates resources to promote these efforts.10 As safety culture improvement has been correlated with improved outcomes, assessing the safety culture and using results to guide education and program development can be used to enhance safety in healthcare systems.11–17
Increasing electronic medical records (EMR) use has been identified as a strategy to improve patient safety.18 EMR systems can help healthcare professionals and institutions organize data and improve workflows; surveys of clinicians who have adopted EMRs in ambulatory care settings support a perception of improved safety.19 However, many clinicians using EMRs often fail to utilize safety improvement functionalities, and EMR implementations can have unintended consequences, adversely impacting quality and safety.20–23 To promote use of such functionalities, the United States government has implemented standards for “meaningful use.”24
In 2007, our organization implemented an integrated EMR as part of ongoing safety and quality improvement efforts. In our post-implementation process, we completed a series of knowledge, quality and safety assessments to identify problems in our new EMR environment, using results to refine training and work processes in our clinical practices. We paid particular attention in our initial survey to primary care providers, due to their roles in patient care and team coordination. In this study, we focus on changes in perceptions of safety among our primary care provider group during the three years following EMR implementation.
Our medical group practice is affiliated with an academic medical center and has central management functions coordinating quality, safety, operations, and information technology. Prior to EMR implementation, our group had electronic prescription management and business systems in place. In 2007, 17 practice sites delivered primary care services to approximately 150,000 patients. Between 2007 and 2009, one practice site split into two offices, for a total of 18 legacy primary care practice sites. Between 2009 and 2010, our organization expanded significantly and added additional practice sites. To compare historically similar units, we included only the 18 legacy practices in this evaluation.
We employed an extensive pre-implementation planning process similar to that described by others.25–28 This process included evaluation of several commercial EMR systems, hardware selection, and secure network installation to connect our practice sites. A medical director for informatics was named, and change champions were identified from multiple levels in our organization. Several new committees were developed, including an executive-level steering committee and a standards workgroup tasked with translating existing quality procedures into EMR workflows. A process for transferring data from the legacy prescribing and business systems was developed and tested.
Implementation of a new EMR (GE Healthcare Centricity) was completed in four phases, with “go-live” occurring between March 2007 and June 2007. The key phase features are shown in Table 1. EMR functionalities believed to improve quality and safety were progressively integrated into workflows, and their use was encouraged with incentives and regular feedback. Since 2007, we have completed two system upgrades; the current system (GE Centricity v9.5) is certified for meaningful use by the Certification Commission for Health Information Technology.
EMR training included a combination of self-study, work with super-users, and on-site training. Immediately prior to go-live, staff attended one four-hour classroom training session. Schedules were reduced to 50 % and then 75 % of baseline during the first two weeks, and on-site “super-user” support was provided. During and after implementation, input from a variety of indicators, including the safety assessments reported below, was used to inform and refine our EMR workflows, training, and support processes.
We assessed safety needs in our practices for the first time at the end of EMR implementation, in June 2007 (T1). We repeated the assessment in January 2009, 1.5 years after implementation (T2); and again in December 2010, approximately 2.5 years (T3) after the initial assessment. Each assessment bundled safety culture surveys and practice-specific needs into a web-based response collector, and links to the collector were distributed by email to employees. The T1 survey was completed using a general survey tool (SurveyMonkey.com, LLC; Palo Alto, CA), and T2 and T3 surveys were completed using a commercial surveyor (Pascal Metrics HealthBench™ Version 1.7; Pascal Metrics Inc., Washington, DC). Our institutional review board approved this study.
Safety culture was assessed with the Safety Attitudes Questionnaire (SAQ), a well-studied and psychometrically validated tool that categorizes organizational safety culture along seven domains: teamwork, safety climate, job satisfaction, stress recognition, working conditions, and perceptions of local and executive management.29 The SAQ rates agreement with a series of statements (e.g., “Medical errors are handled appropriately in this office”) on a five-point Likert scale, with both “agree” and “strongly agree” counting as a positive response. Positive responses are reported as a percentage of all responses. The percentage of positive responses in each domain suggests actionable items, with <60 % indicating “high risk” areas, and 60 % to ≤80 % suggesting areas that need attention.
Minor changes in SAQ question wording, as described by others, were made to tailor the survey to our ambulatory practices.30 Twenty-nine questions were common to all SAQ surveys. (Table 2) Responses were compared directly to evaluate our safety culture over time.
Each survey was supplemented by a practice-specific needs assessment, developed by quality and safety leadership to assess actionable issues specific to our practices at a more detailed level than permitted by the SAQ. Issues presented through a variety of channels, including safety reports, committee input, and informal communications. Because most leaders were super-users, we directly observed how clinicians used and misused the EMR. Some practice-specific needs assessment questions were common to all surveys. For example, each survey requested respondents to indicate their greatest safety concern. In T1, the question was phrased “Of the issues listed below, which one do you think has the most important impact on safety.” In T2 and T3, that question was phrased as “Of the (safety) issues listed, which one do you think needs management attention the most.” In T2 and T3, the practice-specific needs assessment asked respondents to rate the EMR impact on safety.
Other practice-specific needs assessment questions were designed to collect detailed data on specific issues and were not repeated. For example, T1 included the open response question, “What can management do to improve safety issues reporting?”, which directly relates to the ‘safety climate’ SAQ domain. We addressed these issues, and monitored the impact of our efforts with the SAQ ‘safety climate’ domain in subsequent assessments.
The percentage of positive responses was calculated using a denominator that excluded non-respondents and respondents answering “not applicable”. To calculate “domain” averages shown in Table 3, the percentage of positive responses in questions comprising that domain were averaged, after correcting for negatively worded questions by subtracting percent agreement from 100 %. We used chi-square test to calculate P-values assuming independent samples from all three years. Since survey data were collected anonymously, we were not able to adjust for the possible within-person correlations across three years.
The response rates and respondent characteristics for our safety assessments are shown in Table 2. Of primary care providers surveyed, 83.7 %, 85.3 %, and 78.5 % completed surveys at T1, T2, and T3, respectively. Despite growth of our primary care provider group during the study period, the distribution of characteristics and specialties was similar in all three surveys. More than 80 % of primary care providers included in the T1 survey were present at T3.
A comparison of positive responses to questions common to the three SAQ surveys is shown in Table 3. There were non-significant incremental improvements for most domains between T1 and T2, and again between T2 and T3. When assessing improvements in the 2.5 years between the surveys, these combined incremental gains resulted in significant improvements in five of seven domains assessed. Between T1 and T3, there were improvements in job satisfaction (74.1 % vs. 86.8 %, P<0.05), perceptions of executive (59.1 % vs. 72.6 %, P<0.05) and local management (76.2 % vs. 86.2 %, P<0.05), safety climate (76.4 % vs. 87.8 %, P<0.05), and teamwork climate (77.4 % vs. 88.9 %, P<0.05). A significant improvement in the working conditions domain occurred between T2 and T3 (74.3 vs. 84.9, P<0.05). There was a non-significant trend towards improvement in the stress domain. Table 3 also shows significant improvements in several sub-domains over the 1.5 years between T1 and T2, particularly within the safety domain, with significantly improved agreement that medical errors were handled appropriately (79.2 % vs. 93.6 %, P<0.05), that reporting was encouraged (65.7 % vs. 92.3 %, P<0.0001), and that it was easy to learn from errors of others (52.9 % vs. 76.9 %, P<0.0001). A graphic representation of safety culture domain changes is shown in Figure 1.
All practice-specific needs assessment surveys asked providers to identify what they perceived as the single most important safety issue affecting their practices. Results are shown in Table 4. Time constraints were the most significant concern (34.0 % in T1, 35.3 % in T2, and 30.8 % in T3), followed by communication problems. Test result tracking (11.0 %, 7.6 %, and 9.0 %, for T1, T2, and T3, respectively) and medication management (2.0 %, 0.0 %, and 3.0 %, for T1, T2, and T3, respectively) were seen as less significant. EMR issues were ranked as “most important” by only 2 % of providers in T1 and 5.3 % in T3, but 16.8 % of providers in T2 identified EMR-related issues as the most important safety concern. Conversely, only 5 % of respondents rated patient adherence as “most important” in T2, with over 20 % ranking it as “most important” in T1 and T3.
The practice-specific needs assessment portion of T2 and T3, but not T1, asked respondents to indicate the extent to which they agreed with the question “Our electronic medical record has improved our ability to provide safe patient care”. Agreement was high in T2 and remained high in T3 (77.9 % and 85.4 %, respectively; P=0.12).
Safety assessments were conducted in conjunction with other analysis to inform decision making and facilitate meaningful use of the EMR; several applications are outlined in Table 5. For example, the practice-specific needs assessment in T1 asked respondents to indicate barriers to safety issues reporting in our new EMR environment. Responses were grouped into actionable themes, such as making it easier to report, providing more training and assuring blamelessness. Because of this, we implemented a system that allowed clinicians to notify management about safety concerns through a new EMR communication pathway immediately after the T1 assessment. Processes to improve test results tracking and workflow monitoring were also informed and guided by these assessments.
Primary care providers are challenged to provide safe and compassionate care to patients while managing large amounts of data in work systems that are often complex and poorly organized.4,5 Broadening EMR use has been suggested as a strategy to improve safety in primary care settings,31,32 but without careful monitoring implementations may have a negative impact on safety.21–23 Safety culture assessments have been used to evaluate and monitor the impact of safety improvement programs in complex settings.12–14,33,34 We used a validated safety culture survey in combination with practice-specific needs assessments to monitor and respond to safety issues after implementing an electronic medical record system. To our knowledge, assessment of safety culture proximate to an EMR implementation in a large primary care group has not previously been reported.
We found sustained improvement in safety culture over the three years that followed EMR implementation (Table 3, Fig. 1). Scores were lowest in the survey that occurred immediately after implementing paperless documentation in our primary care practices (T1). Comparison of responses in our three safety culture surveys showed improvement within its domains, which were significant between in all areas except stress recognition. In addition, the majority of our primary care providers agreed that the EMR itself enhanced their ability to provide care more safely.
Prior to EMR implementation, our organization did not regularly conduct safety culture assessments. As others have shown, adoption of a periodic safety culture assessment process can raise safety awareness in healthcare settings, and thereby contribute to ongoing improvements.13,14 The goal of improving safety was broadly communicated as one of our medical group practice‘s reasons for adopting the new EMR. The communications, training, and management processes that were integral to our EMR implementation plan, and assured that our workforce used the EMR consistently, would have directly impacted many of the domains measured by the safety culture assessment tool.
We believe that early adoption of many safety-enhancing EMR features also promoted safety culture improvements and the providers’ positive response to the EMR. Prior to implementation, most prescribing had been done in our legacy system, and this continued after transitioning to the new EMR. Electronic laboratory reports were available within our new EMR prior to T1. Further, at the time of the T1 survey, more than 90 % of active patients had medication and problem lists in the EMR, thanks to the data transfer from our legacy systems; virtually all active patients had these in place by T2. Regular feedback of quality and safety data to providers began as early as T1 and expanded during T2 and T3 intervals.
The practice-specific needs assessments were developed by management to characterize emerging safety issues, and complemented the SAQ in helping us define and respond to specific problems. For example, after providers indicated barriers to safety issues reporting in T1, we developed a new reporting system and addressed training between T1 and T2. We were able to monitor the impact of this change by following elements of the ‘safety climate’ SAQ domain, noting significant and sustained improvements. Emerging concerns about workflow management and test results tracking were addressed similarly. We continue to monitor perceptions of “the most important safety issue” in our practice-specific needs assessment and believe that this helps focus management efforts.
As shown in Table 3, time constraints and communications issues were consistently perceived as important safety issues by our providers. Relatively few staff identified medication management as ”the most important safety issue,” which contrasts with observations that medication management is a frequent source of medical error.3,5,35 There are several possible explanations for this difference. Our survey language may have mitigated the intention of assessing “medication errors” (the option was “medication reconciliation” in T1 and “medication lists” in T2 and T3), or our providers may have under-estimated the significance of this issue. However, we believe these responses reflect the impact of electronic prescription management systems in the EMR. We observed in T2 that concerns with “EMR issues” exceeded concerns with patient adherence. We are uncertain about the significance of this finding, but anecdotally noted around that time a tendency of providers to attribute safety issues broadly to “the EMR.”
Our study is subject to several limitations. This was an observational study, and there was no control group. We cannot determine what part of the culture change was related to the EMR, and what part related to the management, communications and educational interventions that accompanied implementation. No data specific to our organization was available prior to T1, which precludes determination of the pre-EMR culture. The T1 assessment may have been negatively biased, due to its delivery immediately following implementation. Additionally, secular attention to enhancing patient safety at national and local levels, or changes outside issues we specifically evaluated, could have caused the culture changes noted during the study period. A statistical limitation is that the response of each individual respondent may have been dependent on his or her first response, but due to de-identification of responses, we were unable to analyze the second administration as a repeat test at the individual level. Further, while a unit level of analysis is typically used for culture analysis, we compared our culture surveys at an organizational level, due to potential provider overlap between practices.36,37 These limitations are mitigated by comparing the same primary care practices over time. Finally, we do not have information on non-respondents since the surveys were conducted anonymously; however, similar respondent characteristics and response rates mitigates this limitation.
We believe that use of the safety culture and practice-specific assessments described above were valuable aids in steering our group’s EMR implementation. While we have not demonstrated patient safety outcomes, other data suggests that improved safety culture is positively correlated with patient safety.12,13,15,16 Implementing regular culture surveys as we adopted our EMR emphasized that safety was an organizational priority. We found that the surveys were useful in “taking the pulse” of our organization during this period of change and subsequent stabilization, and sequential results suggested we were on track with our overall goals of improving safety. Our experience with this safety assessment process may help others who are working through transitions into electronic medical record systems.
We would like to acknowledge our implementation and management teams for major contributions to the strategies listed in Table 1. Medical editor Michael Linde assisted in the preparation of this manuscript.
The authors declare that they do not have a conflict of interest.