Search tips
Search criteria 


Logo of jamiaAlertsAuthor InstructionsSubmitAboutJAMIA - The Journal of the American Medical Informatics Association
J Am Med Inform Assoc. 2011 Nov-Dec; 18(6): 812–819.
Published online 2011 June 22. doi:  10.1136/amiajnl-2010-000018
PMCID: PMC3197984

ICU nurses' acceptance of electronic health records



To assess intensive care unit (ICU) nurses' acceptance of electronic health records (EHR) technology and examine the relationship between EHR design, implementation factors, and nurse acceptance.


The authors analyzed data from two cross-sectional survey questionnaires distributed to nurses working in four ICUs at a northeastern US regional medical center, 3 months and 12 months after EHR implementation.


Survey items were drawn from established instruments used to measure EHR acceptance and usability, and the usefulness of three EHR functionalities, specifically computerized provider order entry (CPOE), the electronic medication administration record (eMAR), and a nursing documentation flowsheet.


On average, ICU nurses were more accepting of the EHR at 12 months as compared to 3 months. They also perceived the EHR as being more usable and both CPOE and eMAR as being more useful. Multivariate hierarchical modeling indicated that EHR usability and CPOE usefulness predicted EHR acceptance at both 3 and 12 months. At 3 months postimplementation, eMAR usefulness predicted EHR acceptance, but its effect disappeared at 12 months. Nursing flowsheet usefulness predicted EHR acceptance but only at 12 months.


As the push toward implementation of EHR technology continues, more hospitals will face issues related to acceptance of EHR technology by staff caring for critically ill patients. This research suggests that factors related to technology design have strong effects on acceptance, even 1 year following the EHR implementation.

Keywords: Human factors, EHR, Geisinger


The push toward implementation of electronic health records (EHR) has raised issues related to the acceptance of the technology.1 This is particularly important in intensive care units (ICUs) where physicians and nurses experience high workload,2 3 patient care is critical and complex,4 5 decisions often need to be made quickly, and interventions must be implemented in a timely manner.4 Any change in the work system of ICUs such as the implementation of EHR technology can have important consequences for providers as well as patients.6 Understanding ICU staff perceptions of the EHR technology and its implementation can help EHR designers and implementers in their continuous effort to improve the design, implementation, and use of the technology. In this study, we report data on EHR acceptance by ICU nurses and analyze factors related to design and implementation of the technology that can contribute to acceptance.


The implementation and use of EHR technology have raised numerous challenges, including end-user acceptance.7 In complex healthcare environments such as ICUs, it is important to understand end-user perceptions of the usability, usefulness, and acceptance of the technology.4 Most research on EHR acceptance has focused on physicians8 9; less is known about nurses' acceptance of EHR technology10 and its different functionalities—for example, the electronic medication administration record (eMAR) or order entry. Some challenges to acceptance and use may be temporary and visible only during the short-term adaptation phase that immediately follows implementation. Other challenges to acceptance and use may arise only after extended use of the technology by users.11 Therefore, we need to examine EHR acceptance at multiple periods of time.12 13

Studies of EHR acceptance have rarely addressed ICU providers, and particularly nurses.10 A small-scale study of nurses' perceptions of EHR was limited to 46 nurses in one medical-surgical unit and one ICU.14 Most nurses (96%) preferred the EHR to paper patient records because it provided enhanced access to patient information, facilitated documentation and information retrieval, and improved organization of work. Because use of the EHR technology by ICU nurses participating in our study is mandatory, continued acceptance of the technology is important15; if nurses find the EHR technology neither usable nor useful and develop negative perceptions of and attitudes toward the EHR technology, it may be difficult to engage them continuously in using the full features of the system and in learning new features of it. Similar negative consequences have been documented in the context of mandatory usage by nurses of bar coding medication administration technology16 and smart intravenous pump technology.17 18 In this context of mandatory technology usage, perceptual measures of acceptance and use are therefore critical as system use data are unlikely to provide information about the system quality.

Various models of technology acceptance have been proposed.7 19 In this study, we draw on two bodies of knowledge to examine EHR acceptance among ICU nurses. First, according to Nielsen,20 technology acceptance is influenced by (1) the usability of the technology (ie, ‘how well users can use the technology functionalities’) and (2) the utility or usefulness of the technology (ie, ‘whether the functionalities of the technology can do what is needed’). (The concepts of usability and usefulness/utility as proposed by Nielsen12 are respectively similar to the concepts of perceived ease of use and perceived usefulness of the Technology Acceptance Model.13) We assume that ICU nurses' acceptance of the EHR technology is influenced by the usability and usefulness of the technology. We asked ICU nurses to evaluate the EHR usability, as well as the usefulness of three EHR functionalities, specifically computerized provider order entry (CPOE), eMAR, and a nursing documentation flowsheet. Second, the manner in which the technology is implemented can influence end-user satisfaction and acceptance.19 21–23 An extensive review of organizational design and management literature shows how characteristics of the technological change process can affect acceptance of the technology.24 A key characteristic of the change process is the ability of end users to participate effectively in the implementation process.24–26 When end users are given a chance to provide input into the design and implementation of the technology, they are more likely to accept and use it.25

The EHR has been defined in many different ways.27 Because EHR technology can include various functionalities, it is important to examine each of them specifically. In this study, we assess ICU nurses' perceptions of the usefulness of three EHR functionalities: CPOE, eMAR, and nursing documentation flowsheets. Our research question is: Do implementation method, technology usability, and usefulness affect nurses' acceptance of the EHR?



This research is part of a larger study investigating the impact of EHR on the work of end users and various outcomes in four ICUs of a regional medical center ( The EHR product studied was the EpicCare Inpatient Clinical System version Spring 2006 (Epic Systems, Madison, Wisconsin). Several functionalities of the EHR were implemented concurrently in October 2007, including CPOE and eMAR. The nurses must use CPOE to review and sign off on entered orders. Verbal orders are entered by nurses but are uncommon. The nurses use the eMAR to review and document medication administration, timing and comments about the administration. The eMAR uses color coding to let nurses know that a medication is currently due or overdue. The nursing flowsheet functionality, in which nurses record information such as vital signs, patient symptoms, and patient care performed, was implemented before the beginning of the study (June 2005), except in the neonatal ICU where it was implemented with the EHR. The EHR system was optimized for each specific ICU. Based on feedback from ICU clinicians (see below the list of implementation activities in which ICU nurses participated) and analysis of ICU care processes performed by the IT team, tools for information display (eg, ‘accordion report’ for presenting complex ICU patient data) were developed and provided coherent views of the complex data typically generated in the care of ICU patients. After implementation, the system evolved incrementally, as care processes, the EHR, and user knowledge, skills, and behaviors were improved. Nurses received 10 h of required competency-based training before implementation. In this paper, we analyze data from two cross-sectional surveys conducted after the October 2007 EHR implementation.

The research site is a 400-bed rural, tertiary care medical center located in the northeastern USA. Nurses in four ICUs were asked to participate in the study: (1) the Adult ICU, which is a 24-bed medical/surgical shock/trauma unit, (2) the 18-bed medical/surgical Cardiac ICU, (3) the 38-bed Neonatal ICU, and (4) the 11-bed Pediatric ICU.

In the two rounds of survey data collection, ICU nurses were invited to complete a questionnaire. Respondents were recruited through multiple means including posters about the study that were displayed in the unit, meetings in each unit describing the study and encouraging participation, email announcements of data collection, and the scheduling of specific dates and times when researchers came to the unit and distributed surveys. Surveys were returned by respondents to a locked mailbox in the unit break room. Researchers went to each ICU several times to continue recruiting participants and to distribute surveys to staff with varied work schedules. Data were collected 3 months (January–February 2008) and 1 year (October 2008) after EHR implementation. Participation was voluntary, and the study was approved by the institutional review boards at the university and the study hospital. The response rate for the 3-month data collection was 51% (121 participating nurses out of 237 eligible participants), while the response rate at 12 months was 72% (161 participating nurses out of 224 eligible participants). The different response rates can be explained by the time and resources involved in the distribution and collection of surveys in each round. For instance, at 12 months, we organized a greater number of meetings in each ICU during which we described the study and more actively recruited nurses for participation in the survey.


The paper questionnaire included items from established instruments to measure technology acceptance, EHR usability, and EHR usefulness.12 29 30 Several questions about the characteristics of respondents were also included, such as their level of participation in implementation and training activities. The questionnaire was pilot-tested with nine end users from nursing units of the medical center other than the ICUs. The objectives of the pilot test were (1) to evaluate the length of the survey and to measure the time needed to complete it; (2) to assess the flow and order of questions; (3) to ensure that the new questions on implementation activities were clear to the respondents; and (4) to gather input from nurses about effective methods for distributing and collecting surveys. In response to pilot test feedback, we eliminated several questions to make the survey shorter, changed the response categories for two questions, clarified the terms referring to specific implementation activities, and refined our recruitment strategy.

A single 10-point Likert-scale item measured the respondents’ overall acceptance of the EHR technology, ranging from (1) ‘dislike very much and don't want to use’ to (10) ‘like very much and eager to use.’ We elected not to use EHR system-usage data for several reasons. First, nurses have to use the EHR technology; therefore, system-usage data may be insensitive to capture the positive and/or negative features of the EHR technology. Second, system-usage data are complex to analyze and interpret.31 32 In particular, recent information systems research indicates the need to examine multiple levels of system use data and their interactions.33 34 Our study focuses on individual nurses' attitudes toward and perceptions of the EHR technology, which are an important focus of research.31 32

Perceived usability measures were selected from the Questionnaire for User Interface Satisfaction (QUIS).30 The seven items of the QUIS scale of learning were combined into a measure of the overall EHR usability by calculating the average response and rescaling it to range from 0 (negative) to 100 (positive).

Perceptions of usefulness were measured for CPOE, the eMAR, and the nursing flowsheet. The items for each functionality were combined into a multi-item scale indicating the perceived usefulness of that functionality. The acceptance and usefulness scales have been used in previous research.12 36 37

In the 3-month postimplementation round, the survey included items on the information received by the end users about EHR implementation and their inputs in decision-making regarding EHR implementation. These items used semantic differential response categories such as ‘vague–precise’ (for information provided to users about the implementation), ‘meaningless–meaningful’ and ‘non-productive–productive’ (for users' opportunities for input) and ‘insufficient–sufficient’ and ‘useless–useful’ (for both). The items were drawn from published literature38 and have been used in previous research.12 37 39 Another set of items asked respondents to indicate whether they participated in 11 EHR implementation activities, such as the health-system-level team, strategic design team, the nurse feedback team, the operations managers' meetings, and the project oversight committee. The implementation activities also included two prospective human factors assessments that were conducted as part of the larger research project, that is, usability evaluation and a proactive risk assessment. The other EHR implementation activities were a multidisciplinary feedback group, departmental meetings, pilot testing, and regulatory reviews.

Questions capturing survey respondent demographics and other characteristics include age, ethnicity, race, clinical work unit, the number of hours typically worked each week, the shift typically worked, years spent working for this hospital, and years spent working in the current ICU. Respondents were also asked to estimate their number of years of computer experience and their years of experience working with the outpatient EHR (which has been used throughout the health system since 2002).


Descriptive analyses were performed to assess nurses' EHR acceptance, the perceived usability and usefulness of the EHR functionalities, and perceptions of the EHR implementation process. Because data were collected with a repeated cross-sectional design, longitudinal analyses could not be performed. Instead, overtime comparisons indicate whether the average responses of nurses differed significantly between short-term (3 months postimplementation) and long-term (12 months postimplementation). Missing data were analyzed using Little's MCAR test40 at each time period. Results indicated that data at 3 months postimplementation were missing 2.95%, with the Little test (χ2=29.12, df=28, p=0.406), and data at 12 months were missing 2.27% with the Little test (χ2=23.56, df=23, p=0.428). Tests indicate that the missing data were random, and no imputation was required.

The model analysis strategy concentrates on covariate influence over time. The analysis of repeated semicross-sectional data from this study poses the potential problem of heteroscedasticity (the violation of the assumption that all residuals are homoskedastic, or have the same variance). This violation would result in biased estimates. To assess the issue of heteroscedasticity in our repeated cross-sectional data, we conducted a two-level hierarchical model for the repeated outcomes.41 42 The model was written as a two-level hierarchical structure, where EHR acceptance is a function of various covariates.

The initial intercepts were modeled as random variables. We treated our two time periods as a repetition at level 1 (indicated by t) nested within nurses (indicated by i). Let zt be a dummy variable of indicator values for each nurse (i), Z1i=1 if t=3 months and 0 otherwise, Z2i=1 if t=12 months and 0 otherwise.

A general multivariate hierarchical model was considered, with the model for these data written as


where xh,ti indicates the covariates (eg, level of computer experience). To assess the issue of heteroscedasticity in our repeated cross-sectional data, we conducted a two-level hierarchical model for the repeated outcomes.

The nurses worked on one of the four ICUs at the two time periods, that is, 3 months and 12 months. Because the analysis initially assumes nurse independence, we assessed the degree of within-ICU dependency. The existence of a non-zero intraclass-ICU correlation coefficient would indicate non-independence in nursing response, which would result in artificially reducing the estimated SE, and bias the interpretation of the significance of a parameter. To assess a cut-off level, we estimated the design effect, which is based on the intra-ICU correlation (1+(average unit size−1)×intraclass correlation). Simulation studies43 have shown that if the estimated design effect is less than 2.00, the bias in SEs due to within-dependency is minimal and does not require adjusting. Finally, parameter contrasts between 3 months and 12 months were conducted based on procedures suggested by Goldstein.41


Description of the sample

Characteristics of the respondents are described in table 1. The respondents in the two rounds differed significantly only in their level of computer experience: nurses in the 12-month round had more computer experience. This difference is controlled for in the acceptance model by including years of computer experience as a covariate.

Table 1
Respondent characteristics in the 3-month and 12-month surveys

Descriptive analyses

Table 2 provides information about basic statistics, Cronbach-α scores and correlations for the study variables. The scales of perceptions of EHR implementation, EHR usability, and usefulness of the three functionalities have a high internal consistency, with Cronbach α scores ranging from 0.90 for usefulness of CPOE to 0.98 for inputs into decisions regarding EHR implementation (see table 2). A factor analysis performed on the usability and usefulness questions showed that the items for each scale load onto single factors that account for between 70% (usability) and 92% (eMAR usefulness) of the total variance.29

Table 2
Reliabilities (α) for scale variables, correlations (r), means and SD (3 months: n=121; 12 months: n=161)

Table 3 shows nurses' perceptions of (1) the information they received about the EHR implementation and (2) the significance of their inputs into implementation decision-making. Nurses' perceptions of the information tend to be slightly positive, as indicated by the mean scale value of 56.7 (from a potential range of 0 to 100). Their ratings of individual scale items are very similar for each measure, ranging from an average of 4.64 for the timeliness of the information to 4.16 for the accuracy of the information (on a scale of 1–7). In contrast, nurses' assessments of the significance of their inputs into decision-making were slightly negative (mean scale value of 41.6). Again, the ratings are similar across the six questions of the scale, ranging from an average of 3.63 for timeliness to 3.38 for meaningfulness. Seventy-one percent of ICU nurses surveyed at 3 months indicated that they never participated in any of the 11 implementation activities. The most frequently reported implementation activities in which ICU nurses participated are nursing feedback team (23%) and departmental meetings (14%).

Table 3
Nurse perceptions of electronic health records (EHR) implementation (3-month data), n=119

Overtime comparisons of nurses' assessments of the acceptance, usability, and usefulness of the EHR technology can be found in figure 1 and table 4. On average, nurses' perceptions were more positive at 12 months than at 3 months. Overall EHR acceptance was slightly positive at 3 months (mean 6.32 on a 10-point scale) and significantly higher at 12 months (mean 6.91). (Note that in figure 1, EHR acceptance was rescaled to range from 0 to 100 to permit comparison with the usability and usefulness scales.) The patterns for usability and usefulness are similar. At 3 months after implementation, nurses had nearly neutral assessments of EHR usability (mean scale value of 49.8 on a 100-point scale), while 1 year after implementation they had slightly positive perceptions (mean scale value of 56.4). The EHR usability items whose average scores significantly improved in this period were related to the straightforwardness of task performance, the helpfulness of screen messages, the degree to which users' needs are taken into consideration, and the ease of correcting mistakes. The usability item related to help messages on screen received the lowest score (mean=4.80 at 3 months) but significantly improved at 12 months. The EHR usability items related to learning to operate the system, exploring new features, and ability to correct one's mistakes received the highest scores (means=6.44, 6.31 and 6.44 respectively at 12 months).

Figure 1
Comparisons of electronic health records (EHR) acceptance, usability and EHR functionality usefulness at 3 months and 12 months after EHR implementation. CPOE, computerized provider order entry; eMAR, electronic medication administration ...
Table 4
Perceived acceptance, usability, and usefulness of the electronic health records (EHR) technology by intensive care unit nurses (mean (SD))

On average, nurses viewed the eMAR and CPOE functionalities as easier and more satisfying to use at 12 months after implementation. The nurses also responded more frequently that the eMAR functions in ways they expect at 12 months. Their perceptions of nursing flowsheet usefulness did not change, perhaps because many of the nurses had been using this system for over 2 years prior to EHR implementation. To test whether the relatively higher level of nurses' computer experience at 12 months was driving the findings of overtime differences in EHR acceptance, EHR usability, CPOE usefulness, and eMAR usefulness, we ran regression analyses (not shown but available from the authors) on variables transformed to correct for skewed distributions. We found that the amount of time since EHR implementation significantly predicted EHR acceptance, EHR usability, and the usefulness of CPOE and eMAR, even when the effect of computer experience was held constant in the model.

Multivariate hierarchical model

The results of the model predicting the level of EHR acceptance in each round can be found in table 5; a graphical representation of the model showing the relationship between EHR acceptance and EHR usability and usefulness is displayed in figure 2. The estimated dependency for ICU based on the design effect was 1.86, indicating no adjustment is required (ICC=0.03, average unit size of 29.75). At both 3 months and 12 months, years of computer experience, work unit, and the respondent's rating of EHR usability and of CPOE usefulness were significant predictors of EHR acceptance. At 3 months postimplementation (but not at 12 months), the rating of eMAR usefulness was also a significant predictor. At 12 months after EHR implementation (but not at 3 months), the rating of nursing flowsheet usefulness was a significant predictor. Time contrasts indicate that the effects of only two predictors, years of computer experience and perceptions of eMAR usefulness, changed significantly between 3 and 12 months. The effect of computer experience on EHR acceptance increased, while the effect of eMAR usefulness declined substantially. The variance estimates indicate that EHR acceptance had more variability at 3 months than at 12 months relative to the predicted variables, but was not significantly different, providing support for the assumption of homoscedasticity. Finally, the R2 estimates indicate that more of the variance in EHR acceptance is explained by the 12-month equation.

Table 5
Model predicting electronic health records (EHR) acceptance by intensive care unit (ICU) nurses, 3 months and 12 months after EHR implementation (3 months: n=121; 12 months: n=161)
Figure 2
Model of electronic health records (EHR) acceptance. CPOE, computerized provider order entry; eMAR, electronic medication administration record.


On average, ICU nurses' acceptance of the EHR technology was rather positive and improved over time. Their average perceptions of EHR usability and the usefulness of CPOE, eMAR, and nursing flowsheets also improved over time. Although the data do not permit longitudinal analyses to be performed, the results suggest that these average improvements in acceptance, usability, and usefulness may be related to the ‘learning curve’ effect. That is, over time (from 3 to 12 months), ICU nurses become more familiar with the EHR technology and its various functionalities because of their increasing use of and exposure to the technology. Nurses may also discover ways that the EHR technology can help them in performing their work, and ways to incorporate EHR use into their workflow or workarounds when using the EHR that improve their workflow efficiency. In addition, during this period of time, physicians may become more proficient at using the EHR, in particular the CPOE functionality, which may reduce the nurses' need to contact physicians about specific orders. Also, post-EHR implementation, the medical center continued to invest in EHR optimization activities. Nurses' inputs regarding improvements in the design and use of the EHR technology were elicited from a focus group of nursing-unit leaders that met weekly.

The improvement in ICU nurses' EHR acceptance found in this study is similar to the improvement in nurses' acceptance of Smart infusion pump technology found in a study of nurses in one academic medical center12: nurses' acceptance of the Smart infusion pump technology was positive (means varying from 6.53 to 7.20 on 10-point scales), and significantly increased between 6 weeks and 1 year after implementation. This improvement was accompanied by numerous increases in the perceived usability of the Smart infusion pump technology, similar to findings in this study.

Only about 30% of ICU nurses participated in at least one of 11 implementation activities, mostly in feedback teams and departmental meetings. This may explain why ICU nurses were more likely to report positive opinions about information received about EHR implementation than about their inputs into decision-making related to EHR implementation. Similar findings have been reported in studies of other technologies, such as Smart infusion pumps.12 EHR implementers should be encouraged to involve nurses in EHR design and implementation activities; participation in those activities can help to improve nurses' perceptions of information received about the implementation and inputs in decisions about the implementation.24 Nurses' participation in the implementation process can also help them to better understand the technology and its functionalities, and their integration in their daily work. Effective and efficient participatory methods (eg, semistructured feedback sessions during scheduled break time) need to be developed to facilitate and support nurses' involvement; this is particularly important for ICU nurses who may not be able to ‘step away’ from patient care.

ICU nurses' perception of overall EHR usability and CPOE usefulness were consistent predictors of EHR acceptance at 3 and 12 months postimplementation. The continued impact of EHR usability and CPOE usefulness on ICU nurses' EHR acceptance emphasizes the need for sustained attention to the design of EHR technology even after implementation.44 45 Because CPOE changes the work of physicians and mid-level providers, that is, the primary intended users of CPOE, changes in nursing workflows enabled by ordering providers' use of CPOE may influence nurses' perceptions of the EHR. One important change is that CPOE requires providers to specify many aspects of orders that were previously specified by nurses, pharmacists, and others. Therefore, nurses may feel less burdened by the need to interpret orders and spend time contacting physicians to clarify orders.

Another change is that providers can enter orders from other locations in the hospital and elsewhere, making verbal orders less necessary, which anecdotal evidence affirms. Physicians are the primary users of the CPOE functionality, and their work was significantly influenced by CPOE implementation.46 Such changes in the work of physicians may indirectly affect nurses' perceptions of CPOE usefulness and their acceptance of EHR technology by making nurses' work more manageable, in both the short- and long term.

Nurses' perceptions of the usefulness of the eMAR and nursing flowsheet had varying impacts on EHR acceptance. At 3 months, nurses who perceived the eMAR as useful were more likely to accept the EHR technology and more eager to use it; however, this was not the case at 12 months. Implementation of the eMAR was a significant change for nurses, who previously used a paper MAR. Therefore, we would expect nurses who perceive the eMAR as useful at 3 months would be more likely to accept the EHR technology and more eager to use it. After a short period of use, nurses understand the benefits of the eMAR functionality, such as access to up-to-date information about medication administrations. This short-term effect of eMAR usefulness on EHR acceptance may disappear at 12 months because, after a transition period, ICU nurses are used to the eMAR, and its usefulness is no longer relevant for their acceptance of the technology; other EHR functionalities may become important over time and influence acceptance. (It is important to note that EHR usability remains a consistent predictor of EHR acceptance over time.) In contrast, nurses' perception of the usefulness of the nursing flowsheet influenced EHR acceptance at 12 months postimplementation but not at 3 months. The nursing flowsheet is a more dynamic functionality of EHR technology. After EHR implementation, iterative changes were made to care processes and the flowsheets that support those processes (eg, addition of flowsheet rows to be completed by nurses), potentially accounting for some of the importance of perceived usefulness of the nursing flowsheets as a determinant of EHR acceptance in the long term (12 months postimplementation). The average perceived usefulness of the nursing flowsheet did not improve between 3 and 12 months after EHR implementation; however, the usefulness ratings are fairly similar to that of the eMAR usefulness.

Study limitations include the sample, as ICU nurses participating in the study came from only one institution that invested significant resources in the design and implementation of the EHR technology. However, this design allowed a deep understanding of the implementation, the characteristics of the EHR technology, and their influence on ICU nurses. The nurses answering the 12-month survey had more computer experience than the nurses who participated in the 3-month post-EHR implementation survey. Given the different response rates at 3 and 12 months, it is possible that nurses who had more computer experience were more likely to participate in the 12-month survey (response rate: 72%) as compared to the 3-month survey (response rate: 51%). The effect of computer experience was entered as a covariate in the models predicting EHR acceptance; at both 3 and 12 months, years of computer experience were positively related to EHR acceptance. As shown in table 5, computer experience was a stronger predictor of EHR acceptance at 12 months than at 3 months. The impact of technology experience on acceptance is well documented.7 47 It is possible that our results are affected by the different survey response rates (51% at 3 months vs 72% at 12 months). However, the only demographic difference that we could identify between nurses who participated in the 3-month survey and nurses who participated in the 12-month survey was for computer experience, and we controlled for this difference in the models.

As the response rates were 51% for the 3-month survey and 72% for the 12-month survey, a selection bias is possible as participating nurses may have more positive perceptions of the EHR technology and its implementation. On the other hand, nurses who had more negative perceptions of the EHR technology and its implementation may have been more likely to participate in the survey in order to express their opinion. However, our results are consistent with other studies of nurses' acceptance of technology, in which increasing use and familiarity with the technology produces an improvement in perceptions of acceptance.12

More research using a longitudinal design is needed to further understand how EHR-related predictors of technology acceptance may change over time.7 This would help implementers of EHR technology identify key issues that need to be addressed in the short- versus long term. In addition, the EHR implementation process can be viewed as continuous,44 involving activities such as optimization of the EHR or software upgrades. These changes over time may influence the usability and usefulness of the EHR and its functionalities, and therefore the level of EHR acceptance by nurses and other users.


As EHRs and other forms of health IT are increasingly being implemented in complex environments such as ICUs, healthcare organizations and designers of the technology need to consider end-user acceptance and the technology-related factors that influence acceptance. Our study shows that ICU nurses' acceptance and perceptions of EHR usability and usefulness improved over a period of 1 year after EHR implementation. EHR usability and the usefulness of CPOE were consistent predictors of EHR acceptance in the short-term (3 months post-EHR implementation) and the long-term (12 months post). The usefulness of the eMAR influenced EHR acceptance at 3 months, whereas the usefulness of nursing flowsheets was a predictor of EHR acceptance at 12 months. Even after 1 year of EHR use, the characteristics of the EHR technology (usability and usefulness) have a significant impact on acceptance and use of the technology by ICU nurses. Therefore, it is important for healthcare organizations to continue their efforts to optimize the design and use of EHR after the technology is implemented.


Funding: This research was made possible by grant R01 HS15274 from the Agency for Healthcare Research and Quality (PI: P Carayon). This publication was also supported by grant 1UL1RR025011 from the Clinical & Translational Science Award (CTSA) program of the National Center for Research Resources in the National Institutes of Health (PI: M Drezner).

Competing interests: None.

Ethics approval: Ethics approval was provided by the UW-Madison Health Sciences IRB.

Provenance and peer review: Not commissioned; externally peer reviewed.


1. Despont-Gros C, Mueller H, Lovis C. Evaluating user interactions with clinical information systems: a model based on human–computer interaction models. J Biomed Inform 2005;38:244–55 [PubMed]
2. Carayon P, Gurses AP. Nursing workload and patient safety in intensive care units: a human factors engineering evaluation of the literature. Intensive Crit Care Nurs 2005;21:284–301 [PubMed]
3. Embriaco N, Papazian L, Kentish-Barnes N, et al. Burnout syndrome among critical care healthcare workers. Curr Opin Crit Care 2007;13:482–8 [PubMed]
4. Clemmer TP. Computers in the ICU: where we started and where we are now. J Crit Care 2004;19:201–7 [PubMed]
5. Garland A. Improving the ICU: part 1. Chest 2005;127:2151–64 [PubMed]
6. Carayon P, Schoofs Hundt A, Karsh BT, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006;15(Suppl I):i50–8 [PMC free article] [PubMed]
7. Holden RJ, Karsh BT. The technology acceptance model: its past and its future in healthcare. J Biomed Inform 2010;43:159–72 [PMC free article] [PubMed]
8. Dansky KH, Gamm LD, Vasey JJ, et al. Electronic medical records: are physicians ready? J Healthc Manag 1999;44:440–55 [PubMed]
9. Yarbrough AK, Smith TB. Technology acceptance among physicians—a new take on TAM. Med Care Res Rev 2007;64:650–72 [PubMed]
10. Oroviogoicoechea C, Elliott B, Watson R. Review: evaluating information systems in nursing. J Clin Nurs 2008;17:567–75 [PubMed]
11. Yeh RKJ, Teng JTC. Extended conceptualisation of perceived usefulness: empirical test in the context of information system use continuance. Behav Inform Tech. In press.
12. Carayon P, Hundt AS, Wetterneck TB. Nurses' acceptance of Smart IV pump technology. Int J Med Inform 2010;79:401–11 [PMC free article] [PubMed]
13. Wall TD, Clegg CW. A longitudinal study of group work redesign. J Occup Behav 1981;2:31–49
14. Kossman SP, Scheidenhelm SL. Nurses' perceptions of the impact of electronic health records on work and patient outcomes. Comput Inform Nurs 2008;26:69–77 [PubMed]
15. Sorebo O, Eikebrokk TR. Explaining IS continuance in environments where usage is mandatory. Comput Human Behav 2008;24:2357–71 [Article].
16. Koppel R, Wetterneck T, Telles JL, et al. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008;15:408–23 [PMC free article] [PubMed]
17. Rothschild J, Keohane CA, Cook EF, et al. A controlled trial of smart infusion pumps to improve medication safety in critically ill patients. Crit Care Med 2005;33:533–40 [PubMed]
18. Leape LL. ‘Smart’ pumps: a cautionary tale of human factors engineering. Crit Care Med 2005;33:679–80 [PubMed]
19. Mahmood MA, Burn JM, Gemoets LA, et al. Variables affecting information technology end-user satisfaction: a meta-analysis of the empirical literature. Int J Hum Comput Stud 2000;52:751–71
20. Nielsen J. Usability Engineering. Amsterdam: Morgan Kaufmann, 1993
21. Korunka C, Zauchner S, Weiss A. New information technologies, job profiles, and external workload as predictors of subjectively experienced stress and dissatisfaction at work. Int J Hum Comput Interact 1997;9:407–24
22. McLane S. Designing an EMR planning process based on staff attitudes toward and opinions about computers in healthcare. Comput Inform Nurs 2005;23:85–92 [PubMed]
23. Lorenzi NM, Riley RT. Managing change: an overview. J Am Med Inform Assoc 2000;7:116–24 [PMC free article] [PubMed]
24. Karsh BT. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care 2004;13:388–94 [PMC free article] [PubMed]
25. Eason K. New systems implementation. In: Wilson JR, Corlett N, editors. , eds. Evaluation of Human Work. 2nd edn London: Taylor & Francis, 1995:1055–70
26. Korunka C, Weiss A, Zauchner S. An interview study of ‘continuous’ implementations of information technologies. Behav Inform Tech 1997;16:3–16
27. Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 2008;77:291–304 [PubMed]
28. Walker JM. Implementation of electronic order management in a hospital's four intensive care units: a structured description of organizational setting and information system implementation. 2010
29. Hoonakker PL, Cartmill RS, Carayon P, et al. Development and psychometric qualities of the SEIPS survey to evaluate CPOE/EHR implementation in ICUs. Int J Healthc Inf Syst Inform 2011;6:51–69 [PMC free article] [PubMed]
30. Chin JP, Diehl VA, Norman KL. Development of an instrument measuring user satisfaction of the human–computer interface. In: Proceedings of ACM SIGCHI. New York: ACM/SIGCHI, 1998:213–18
31. Straub D, Limayem M, Karahanna-Evaristo E. Measuring system usage: implications for IS theory testing. Manag Sci 1995;41:1328–42
32. Rice RE, Borgman CL. The use of computer-monitored data in information science and communication research. J Am Soc Inf Sci 1983;34:247–56
33. Burton-Jones A, Straub DW. Reconceptualizing system usage: an approach and empirical test. Inf Syst Res 2006;17:228–46
34. Burton-Jones A, Gallivan MJ. Toward a deeper understanding of system usage in organizations: a multilevel perspective. MIS Q 2007. Dec;31(4):657–79
35. Chin JP, Diehl VA, Norman KL. Development of an instrument measuring user satisfaction of the human–computer interface. In: Proceedings of SIGCHI. New York: ACM/SIGCHI, 1998:213–18
36. Wetterneck TB, Carayon P, Hundt AS, et al. Nurses' perception of smart IV pump technology characteristics and quality of working life. In: Pikaar RN, Koningsveld EA, Settels PJ, editors. , eds Proceedings of the IEA2006 Congress. Elsevier, 2006
37. Wetterneck TB, Carayon P, Sobande F, et al., editors. , eds. Technology Characteristics Predicting End User Acceptance of Smart Intravenous Infusion Pumps. Orlando, FL: The Human Factors and Ergonomics Society, 2005
38. Bailey JE, Pearson SW. Development of a tool for measuring and analyzing computer user satisfaction. Manag Sci 1983;29:530–45
39. Carayon P, Smith P, Hundt AS, et al. Implementation of an Electronic Health Records system in a small clinic. Behav Inform Tech 2009;28:5–20
40. Little RJA. A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc 1988;83:1198–202
41. Goldstein H. Multilevel Statistical Models. London: Arnold, 2003
42. Yang M, Heath A, Goldstein H. Multilevel models for repeated binary outcomes: attitudes and vote over the electoral cycle. Journal of the Royal Statistical Society 2000;163:49–62
43. Muthen BO, Satorra A. Complex sample data in structural equation modeling. Sociol Methodol 1995;25:267–316
44. Weick KE, Quinn RE. Organizational change and development. Annu Rev Psychol 1999;50:361–86 [PubMed]
45. Cornford T. Information systems and new technologies: taking shape in use. In: Avgerou C, La Rovere R, editors. , eds. Information Systems and the Economics of Innovation. London: Elgar, 2003
46. Hoonakker PL, Carayon P, Walker JM. Measurement of CPOE end-user satisfaction among ICU physicians and nurses. Appl Clin Inform 2010;1:268–85 [PMC free article] [PubMed]
47. Ammenwerth E, Mansmann U, Iller C, et al. Factors affecting and affected by user acceptance of computer-based nursing documentation: results of a two-year study. J Am Med Inform Assoc 2003;10:69–84 [PMC free article] [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association