On average, ICU nurses' acceptance of the EHR technology was rather positive and improved over time. Their average perceptions of EHR usability and the usefulness of CPOE, eMAR, and nursing flowsheets also improved over time. Although the data do not permit longitudinal analyses to be performed, the results suggest that these average improvements in acceptance, usability, and usefulness may be related to the ‘learning curve’ effect. That is, over time (from 3 to 12 months), ICU nurses become more familiar with the EHR technology and its various functionalities because of their increasing use of and exposure to the technology. Nurses may also discover ways that the EHR technology can help them in performing their work, and ways to incorporate EHR use into their workflow or workarounds when using the EHR that improve their workflow efficiency. In addition, during this period of time, physicians may become more proficient at using the EHR, in particular the CPOE functionality, which may reduce the nurses' need to contact physicians about specific orders. Also, post-EHR implementation, the medical center continued to invest in EHR optimization activities. Nurses' inputs regarding improvements in the design and use of the EHR technology were elicited from a focus group of nursing-unit leaders that met weekly.
The improvement in ICU nurses' EHR acceptance found in this study is similar to the improvement in nurses' acceptance of Smart infusion pump technology found in a study of nurses in one academic medical center12
: nurses' acceptance of the Smart infusion pump technology was positive (means varying from 6.53 to 7.20 on 10-point scales), and significantly increased between 6 weeks and 1 year after implementation. This improvement was accompanied by numerous increases in the perceived usability of the Smart infusion pump technology, similar to findings in this study.
Only about 30% of ICU nurses participated in at least one of 11 implementation activities, mostly in feedback teams and departmental meetings. This may explain why ICU nurses were more likely to report positive opinions about information received about EHR implementation than about their inputs into decision-making related to EHR implementation. Similar findings have been reported in studies of other technologies, such as Smart infusion pumps.12
EHR implementers should be encouraged to involve nurses in EHR design and implementation activities; participation in those activities can help to improve nurses' perceptions of information received about the implementation and inputs in decisions about the implementation.24
Nurses' participation in the implementation process can also help them to better understand the technology and its functionalities, and their integration in their daily work. Effective and efficient participatory methods (eg, semistructured feedback sessions during scheduled break time) need to be developed to facilitate and support nurses' involvement; this is particularly important for ICU nurses who may not be able to ‘step away’ from patient care.
ICU nurses' perception of overall EHR usability and CPOE usefulness were consistent predictors of EHR acceptance at 3 and 12 months postimplementation. The continued impact of EHR usability and CPOE usefulness on ICU nurses' EHR acceptance emphasizes the need for sustained attention to the design of EHR technology even after implementation.44
Because CPOE changes the work of physicians and mid-level providers, that is, the primary intended users of CPOE, changes in nursing workflows enabled by ordering providers' use of CPOE may influence nurses' perceptions of the EHR. One important change is that CPOE requires providers to specify many aspects of orders that were previously specified by nurses, pharmacists, and others. Therefore, nurses may feel less burdened by the need to interpret orders and spend time contacting physicians to clarify orders.
Another change is that providers can enter orders from other locations in the hospital and elsewhere, making verbal orders less necessary, which anecdotal evidence affirms. Physicians are the primary users of the CPOE functionality, and their work was significantly influenced by CPOE implementation.46
Such changes in the work of physicians may indirectly affect nurses' perceptions of CPOE usefulness and their acceptance of EHR technology by making nurses' work more manageable, in both the short- and long term.
Nurses' perceptions of the usefulness of the eMAR and nursing flowsheet had varying impacts on EHR acceptance. At 3 months, nurses who perceived the eMAR as useful were more likely to accept the EHR technology and more eager to use it; however, this was not the case at 12 months. Implementation of the eMAR was a significant change for nurses, who previously used a paper MAR. Therefore, we would expect nurses who perceive the eMAR as useful at 3 months would be more likely to accept the EHR technology and more eager to use it. After a short period of use, nurses understand the benefits of the eMAR functionality, such as access to up-to-date information about medication administrations. This short-term effect of eMAR usefulness on EHR acceptance may disappear at 12 months because, after a transition period, ICU nurses are used to the eMAR, and its usefulness is no longer relevant for their acceptance of the technology; other EHR functionalities may become important over time and influence acceptance. (It is important to note that EHR usability remains a consistent predictor of EHR acceptance over time.) In contrast, nurses' perception of the usefulness of the nursing flowsheet influenced EHR acceptance at 12 months postimplementation but not at 3 months. The nursing flowsheet is a more dynamic functionality of EHR technology. After EHR implementation, iterative changes were made to care processes and the flowsheets that support those processes (eg, addition of flowsheet rows to be completed by nurses), potentially accounting for some of the importance of perceived usefulness of the nursing flowsheets as a determinant of EHR acceptance in the long term (12 months postimplementation). The average perceived usefulness of the nursing flowsheet did not improve between 3 and 12 months after EHR implementation; however, the usefulness ratings are fairly similar to that of the eMAR usefulness.
Study limitations include the sample, as ICU nurses participating in the study came from only one institution that invested significant resources in the design and implementation of the EHR technology. However, this design allowed a deep understanding of the implementation, the characteristics of the EHR technology, and their influence on ICU nurses. The nurses answering the 12-month survey had more computer experience than the nurses who participated in the 3-month post-EHR implementation survey. Given the different response rates at 3 and 12 months, it is possible that nurses who had more computer experience were more likely to participate in the 12-month survey (response rate: 72%) as compared to the 3-month survey (response rate: 51%). The effect of computer experience was entered as a covariate in the models predicting EHR acceptance; at both 3 and 12 months, years of computer experience were positively related to EHR acceptance. As shown in , computer experience was a stronger predictor of EHR acceptance at 12 months than at 3 months. The impact of technology experience on acceptance is well documented.7
It is possible that our results are affected by the different survey response rates (51% at 3 months vs 72% at 12 months). However, the only demographic difference that we could identify between nurses who participated in the 3-month survey and nurses who participated in the 12-month survey was for computer experience, and we controlled for this difference in the models.
As the response rates were 51% for the 3-month survey and 72% for the 12-month survey, a selection bias is possible as participating nurses may have more positive perceptions of the EHR technology and its implementation. On the other hand, nurses who had more negative perceptions of the EHR technology and its implementation may have been more likely to participate in the survey in order to express their opinion. However, our results are consistent with other studies of nurses' acceptance of technology, in which increasing use and familiarity with the technology produces an improvement in perceptions of acceptance.12
More research using a longitudinal design is needed to further understand how EHR-related predictors of technology acceptance may change over time.7
This would help implementers of EHR technology identify key issues that need to be addressed in the short- versus long term. In addition, the EHR implementation process can be viewed as continuous,44
involving activities such as optimization of the EHR or software upgrades. These changes over time may influence the usability and usefulness of the EHR and its functionalities, and therefore the level of EHR acceptance by nurses and other users.