4.1 Perceptions of technology implementation, technical performance and usability
Data on technology implementation, technical performance of the Smart IV pump, and usability of the Smart IV pump are provided in to . The MANOVA’s for each of the 6 groups of variables were all found to be statistically significant (p<.001) (see , and for additional information on the MANOVA’s). The results reported in the rest of this section are based on the Bonferroni post-hoc tests.
User Expectations and Perceptions of Smart IV Pump Technology Implementation
Perceived Usability of the Smart IV Pump
Perceived Technical Performance of the Smart IV Pump
In the pre-implementation survey and 6-week-post-implementation survey, on average, respondents reported somewhat positive perceptions of information received about the Smart IV pump implementation (means varying from 4.96 to 5.52 on 7-point scales); their perceptions of user inputs in decision making on pump implementation were somewhat negative on average (means varying from 3.76 to 4.13 on 7-point scales) (see ). One measure - usefulness of information received about pump implementation - was found to be significantly different in the pre- and post-implementation surveys (p<.05). Respondents rated the information they received more useful before the implementation than six weeks after implementation. In the three surveys, there was a significant difference in user perceptions of supplemental training materials (p<.01). Respondents reported that the training materials were more confusing in the 6-week and 1-year-post-implementation surveys (p<.01 for both comparisons) than before implementation.
In the three surveys, respondents reported somewhat positive perceptions of the technical performance of the Smart IV pump (means varying from 5.16 to 6.55 on 10-point scales), except for the two measures added in the one-year-post-implementation survey about pump noise (see ). Because nursing staff had verbally complained of “annoying” air-in-line alarms and beeps resulting from a delay on a continual basis from the time of implementation, these two questions were added. User experience with pump reliability and noise significantly changed from pre- to one-year-post-implementation (p<.05 and p<.001, respectively). Respondents’ perception of pump reliability one year after implementation was significantly lower than before implementation (p<.05). Compared to their expectations of pump noise before implementation, more negative responses were given in both the 6-week-post-implementation survey and the 1-year-post-implementation survey (p<.001 for both comparisons).
Responses to questions on learnability of the Smart IV pump were generally somewhat positive (means varying from 5.33 to 6.55 on 10-point scales) (see ). Learning to operate the pump became easier one year after implementation, compared to either before or six weeks after implementation (p<.001 for both comparisons). Responses to the item “pump is designed for all levels of users” were more positive one year after implementation than before or six weeks after implementation (p<.05 for both comparisons).
Responses to the questions on memorability of the Smart IV pump were somewhat positive (means varying from 5.38 to 5.66 on 10-point scales) (see ). No significant difference was found from pre- to 1-year-post-implementation.
Across the three surveys, responses to the questions on efficiency of using the Smart IV pump were mostly somewhat positive (means varying from 3.95 to 5.36 on 7-point scales) (see ). Responses tended to be positive for two questions on efficiency related to the pump in general improving patient safety (means varying from 4.87 to 5.36) and for the specific question on the drug libraries increasing safety and quality (mean 5.30), and less positive for the question on ease of use of the pump in emergency situations (mean: 4.03) (Note: this question was added in the one-year-post-implementation survey). For seven out of eight questions on efficiency, perceptions improved from the six-week post implementation to one-year-post-implementation: “enables me to accomplish task more quickly” (p<.001), “improves the quality of care I provide” (p<.05), “improves the safety of care I provide” (p<.01), “enhances my effectiveness on the job” (p<.01), “makes it easier to do my job” (p<.001), “increases the safety of care provided to our patients” (p<.001), and “pump functions as I expect” (p<.01). Responses to the question “task can be performed in a straightforward manner” did not change over time.
In the three surveys, respondents’ ratings of alarm and alert messages, and error recovery were also somewhat positive (means varying from 5.18 to 6.18 on 10-point scales) (see ). Six weeks after implementation, the respondents reported less positive perceptions of alarm messages for pump functioning than they expected before implementation (p<.01), which remained less positive in the 1-year-post-implementation surveys. User perceptions of alert messages for the drug library became significantly more positive one year after implementation, compared to six weeks after implementation (p<.001).
Respondents’ satisfaction with the Smart IV pump was addressed by the ratings of their interaction with the pump (see ). Nurses reported easier interaction with the pump one year after implementation, compared to either before or six weeks after implementation (p<.001 for both comparisons). They also reported that the pump was more rigid six weeks after implementation than what they expected before implementation (p<.05); however, the perceptions returned to pre-implementation levels one year after implementation.
4.2 User Acceptance of Smart IV Pump Technology
Data on user acceptance across the three surveys are displayed in . User acceptance of the Smart IV pump technology was positive (means varying from 6.53 to 7.20 on 10-point scales), and significantly increased one year after implementation as compared to six weeks after implementation (p<.001).
User Acceptance of Smart IV Pump Technology
4.3 Predictors of user acceptance
shows the results of the regression analysis performed to examine the predictors of user acceptance as measured at the one-year post-implementation survey. The five groups of predictors were significantly related to user acceptance (adjusted R2 from 26% for learnability and memorability to 64% for efficiency). With regard to technical performance, nurses who reported the pump to be reliable, to have sufficient programming speed, to produce useful alert messages for drug library, and to be quiet were more likely to accept the pump. Out of 6 questions on learnability and memorability, only the question on “designed for all levels of users” was a significant predictor of user acceptance. Five of 10 questions on efficiency were significant predictors of user acceptance. Nurses who reported more positive perceptions of the pump’s efficiency (e.g., pump making one’s job easier to do, pump functioning as expected) were more likely to accept the pump. Four of the five questions on errors and all of the questions on satisfaction were significant predictors of user acceptance. When nurses reported that it was easy to correct mistakes, that using drug libraries helped prevent medication errors, that alarm messages for pump functioning were acceptable, and that the pump was able to identify a problem and provide an appropriate alarm, their level of pump acceptance was higher.
Predictors of User Acceptance – Results of Stepwise Regression Analysis (one-year post-implementation survey)