|Home | About | Journals | Submit | Contact Us | Français|
To identify predictors of nurses' acceptance of bar coded medication administration (BCMA).
Cross-sectional survey of registered nurses (N=83) at an academic pediatric hospital that recently implemented BCMA.
Surveys assessed seven BCMA-related perceptions: ease of use; usefulness for the job; social influence from non-specific others to use BCMA; training; technical support; usefulness for patient care; and social influence from patients/families. An all possible subset regression procedure with five goodness-of-fit indicators was used to identify which set of perceptions best predicted BCMA acceptance (intention to use, satisfaction).
Nurses reported a moderate perceived ease of use and low perceived usefulness of BCMA. Nurses perceived moderate-or-higher social influence to use BCMA and had moderately positive perceptions of BCMA-related training and technical support. Behavioral intention to use BCMA was high, but satisfaction was low. Behavioral intention to use was best predicted by perceived ease of use, perceived social influence from non-specific others, and perceived usefulness for patient care (56% of variance explained). Satisfaction was best predicted by perceived ease of use, perceived usefulness for patient care, and perceived social influence from patients/families (76% of variance explained).
Variation in and low scores on ease of use and usefulness are concerning, especially as these variables often correlate with acceptance, as found in this study. Predicting acceptance benefited from using a broad set of perceptions and adapting variables to the healthcare context.
Success with BCMA and other technologies can benefit from assessing end-user acceptance and elucidating the factors promoting acceptance and use.
The Institute of Medicine estimates an incidence of one medication error per day for the average hospitalized patient.1 The medication administration stage accounts for 26%–32% of all adult patient medication errors2 3 and up to 60% of pediatric patient medication errors.4 Medication administration errors are especially concerning because they have a very low (0%–2%) chance of being intercepted before resulting in patient harm, compared to errors in other stages (eg, 48%–49% chance of medication ordering errors being intercepted).3 5 Bar coded medication administration (BCMA) technology is recommended1 6–9 and increasingly implemented by US hospitals10 as one way to prevent and intercept medication administration errors.11
Several studies report the medication safety benefits of BCMA,12–14 although these benefits are not universal.15 16 However, the mere presence of BCMA or any other health information technology (IT) is not sufficient to reap its potential safety benefits: clinicians must also accept and appropriately use available IT.17–20 There is evidence that BCMA systems are not always favorably perceived by workers,21 22 are sometimes resisted,23 and are worked around or used inappropriately.24–30 To design and implement BCMA systems that clinicians will be more likely to accept and use appropriately, it is important to understand what shapes BCMA acceptance and use.
The objective of this study was to model BCMA acceptance in a sample of registered nurses at a pediatric hospital that recently implemented BCMA. Using survey data, we conducted analyses to identify which combination of perceptions best predicted BCMA acceptance.
More generally, this study sought to add to the growing recognition of, and literature on, implementation science in the context of health IT.20 31–42 Implementation science literature seeks to understand the who, what, when, where, and why of health IT implementation, as well as the predictors, moderators, and mediators of health IT implementation success or failure. Outside of healthcare, research on technology implementation can trace its roots to pioneering work on organizational change,43 44 although current research on technology implementation has been largely driven by research on technology acceptance,45–49 and technology adoption and diffusion.35 50–53 This study contributes to health IT implementation science by comparing theory-driven quantitative models of health IT acceptance.
The study's overarching conceptual framework was based on the Technology Acceptance Model (TAM), a family of theories—TAM,46 TAM2,47 and TAM354—that posit a causal relationship (a) between user perceptions of technology and user acceptance and (b) between acceptance and actual use. TAM is the most widely used theoretical framework in the health IT acceptance and use literature, although no previous work has used TAM to study nurses’ BCMA acceptance.31 55 Holden and Karsh31 reviewed TAM research in healthcare and concluded that TAM predicts health IT acceptance and use reasonably well but proposed two important future directions for modeling health IT acceptance (figure 1):
Accordingly, we measured nurses' perceptions of BCMA using (a) both the variables from the base TAM as well as additional theoretically important variables and (b) both generic and contextualized versions of these variables. We then constructed two statistical models to identify which combinations of variables most efficiently predicted BCMA acceptance.
For Model 1, the outcome of acceptance was operationalized as nurses' behavioral intention (BI) to use BCMA.46 56 BI is the measure of acceptance specified in TAM, is conceptually similar to an individual's motivation to use technology, and has been shown to correlate with actual use.57 For Model 2, we used an alternative measure of acceptance: nurses' satisfaction (SATISF) with BCMA. Organizations usually mandate the use of BCMA and other safety-critical health IT. Technology acceptance research suggests that when technology use is mandatory, SATISF may be a better indicator of acceptance than BI.58 SATISF is an overall evaluation or attitude toward BCMA, has been found to predict actual use, and is a classic conceptualization of acceptance.59–61
For both models, nine theory-based variables formed the set of candidate predictors of acceptance: seven perception and two demographic variables. The first variable was generic perceived ease of use (PEOU), or ‘the degree to which a person believes that using a particular system would be free of effort’ (Davis, p 320).46 The second was generic perceived usefulness (PU), or ‘the degree to which a person believes that using a particular system would enhance his or her job performance’ (Davis, p 320).46 These are the two key perceptions predicting acceptance in the parsimonious base TAM. We also included several perception variables reflecting recent research extending the base TAM.47 48 54 Thus, the third predictor variable was perceived social influence (SI), or ‘the degree to which an individual perceives that important others believe he or she should use the new system’ (Venkatesh et al, p 451).48 The fourth and fifth variables represented perceived facilitating conditions, or ‘the degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system’ (Venkatesh et al, p 453).48 Specifically, these were: perceptions of BCMA training received (TRAINING) and perceptions of technical support (SUPPORT). The two remaining perception variables were generic TAM variables adapted to the healthcare context. We adapted generic PU, which refers to IT's usefulness for the clinician's job,31 to form a contextualized sixth predictor variable representing the perceived usefulness of BCMA for improving patient care (PU-PT). We also adapted generic SI, which refers to the perceived social influence of non-specific ‘important others,’ to form a contextualized seventh predictor variable representing the perceived social influence of patients (or their families) on nurses to use BCMA (SI-PT).62 The eighth and ninth predictors were two demographic variables found to influence acceptance in recent TAM research47 48 54 63: age (AGE) and experience (EXP). All the variables are depicted in figure 1.
We hypothesized that BCMA acceptance (BI, SATISF) would be best predicted by a larger set of variables than the two (PEOU and PU) found in the base TAM and that contextualized variables (PU-PT and SI-PT) would be better predictors of acceptance than their generic counterparts (PU and SI).
This study was a cross-sectional segment of a larger, longitudinal study of the impact of BCMA on patient and employee safety. For this study, data were taken from paper surveys administered to ‘full time’ (≥24 h/week) registered nurses at a 236-bed freestanding pediatric academic hospital in the Midwest US. Nurses from three care units were surveyed: a pediatric intensive care unit; a hematology/oncology/bone marrow transplant unit; and a medical/surgical unit. The survey was administered in August–November 2007, on average 3 months after study units began using BCMA (Centricity Pharmacy; GE Healthcare, Wauwatosa, Wisconsin, USA).i Box 1 contains more information about the hospital, units, participants, and BCMA system.
Eligible registered nurses providing direct patient care were informed about the study and hand-delivered survey packets during in-services, staff meetings, or shift-change meetings. Each survey packet was individually labeled with a nurse's name and a unique ID; each contained a personalized cover letter, survey, informational sheet/consent form, stamped reply envelope, and US$5 cash incentive. Nurses were instructed to complete the survey on their own time. Reminder postcards were placed in nurse mailboxes after 1 week, followed by a new survey packet 7–10 days thereafter, and another reminder postcard 7–10 days later. The study was approved by human subjects/Institutional Review Boards at the University of Wisconsin–Madison and at the research hospital.
Standard multi-item scales validated in prior studies were used for PEOU,49 PU,49 SI,64 TRAINING,65 66 and SUPPORT.65 66 BI was measured with two items from Venkatesh and Morris67 and two items created for the study. The more contextualized PU-PT, SI-PT, and SATISF scales were newly created for the study based on research literature. Frequent prompts in the survey asked nurses to think of ‘the past 30 days’ when responding. Responses were on a numbered seven-point scale ranging from 0 to 6, with the response category labels ‘not at all,’ ‘a little,’ ‘some,’ ‘a moderate amount,’ ‘pretty much,’ ‘quite a lot,’ and ‘a great deal,’ as well as an option to mark ‘don't know.’ AGE was self-reported within five categories (20–29, 30–39, … 60+). EXP was measured as nurses' self-reported job tenure, in years and months.
All survey items underwent extensive evaluation procedures,68 69 including cognitive interviewing70 with 16 non-study nurses. The scales' psychometric properties are reported in table 1. Component items are reported in the Results section.
Scales were constructed by taking the mean (floating denominator) of all valid individual item responses.
We used an all possible subset regression procedure to investigate the optimal subset of predictors (regressors) for BI (Model 1) and SATISF (Model 2). Candidate predictors were PEOU, PU, SI, TRAINING, SUPPORT, PU-PT, SI-PT, AGE, and EXP.ii Models of every combination of predictors were then assessed using five goodness-of-fit measures: (1) R2; (2) root mean square error (RMSE); (3) Mallow's Cp statistic; (4) Akaike information criterion (AIC); and (5) Bayesian information criterion (BIC). The optimal model was considered when higher R2 values were associated with low RMSE, low AIC and BIC, and a Mallow's Cp less than the number of predictors plus 1. Various model comparisons were then conducted based on these five fit measures.
We used Stata (StataCorp, LP) for analyses. An a priori α criterion of 0.05 was used and 95% CIs were calculated around parameter estimates.
Ninety-four of 202 nurses (46.5%) returned a survey. Eleven nurses were excluded from final analysis because they reported working <24 h/week, yielding an analyzed data set of N=83.
Table 2 reports nurse respondents' mean scores on the seven BCMA perception scales as well as response frequencies for individual scale items. Perception scores were positively correlated with one another (all p's <0.05).
Overall, nurses rated the BCMA system as moderately easy to use (PEOU, mean 2.80, SD 1.17). Thirty percent reported that BCMA was ‘not at all’ or only ‘a little’ easy to use and 31% perceived that using BCMA required ‘quite a lot’ of mental effort or more.
Nurses rated BCMA as only somewhat useful for improving either personal job performance (PU, mean 1.82, SD 1.45) or patient care (PU-PT, mean 1.92, SD 1.46). Over 20% perceived that BCMA did ‘not at all’ improve job performance and was ‘not at all’ useful in their job; even larger proportions of nurses did not perceive BCMA at all improving productivity or effectiveness. Over half the nurses perceived that BCMA did ‘not at all’ make patient care easier, although 55% perceived that BCMA reduced the likelihood of medication error ‘a moderate amount’ or more.
Nurses reported high social influence from non-specific ‘others’ (SI, mean 4.09, SD 1.66) and moderate-or-higher social influence from patients/families (SI-PT, mean 3.35, SD 1.54). In response to the question, ‘To what extent do people who influence your behavior think that you should use the bar coding system?’ 58% responded ‘quite a lot’ or ‘a great deal.’ Although 75% reported that patients or their families perceived ‘a moderate amount’ or more that BCMA is good for quality patient care, 26% perceived that patients or their families appreciated being scanned before medication administration ‘a little’ or ‘not at all.’
Perceptions of training (TRAINING, mean 2.77, SD 1.26) and technical support (SUPPORT, mean 3.11, SD 1.43) were centered about the midpoint. Nurses' perceptions of the timing of training were more favorable; perceptions of the completeness of training were less favorable.
Table 3 reports mean scores and response frequencies for questions about BCMA acceptance. BI and SATISF were positively correlated (r=0.81, p<0.05).
Intention to use BCMA was relatively high (BI, mean 3.80, SD 1.22). However, 28% reported that if it were up to them, they would not use BCMA at all, suggesting that when alternatives (eg, workarounds) are available, nurses might not use BCMA. Satisfaction with BCMA was low (SATISF, mean 2.00, SD 1.47). Over a third of respondents reported that they would not recommend the BCMA system to a friend at another hospital and 70% reported ‘a moderate amount’ or more dissatisfaction with BCMA (100% reported being at least somewhat dissatisfied).
Figure 2 depicts the best subset of predictors of BI and SATISF. BI was best predicted by a combination of PEOU, SI, and PU-PT (adjusted R2=0.56±0.14; AIC 183.9; BIC 193.1). A model replacing PU-PT with generic PU provided a worse fit (adjusted R2=0.52±0.15; AIC 191.5; BIC 200.8), confirming the benefit of using the contextualized PU-PT.
SATISF was best predicted by a combination of PEOU, PU-PT, and SI-PT (adjusted R2=0.76±0.09; AIC 165.0; BIC 174.2). A model replacing PU-PT with generic PU and replacing SI-PT with generic SI provided a worse fit (adjusted R2=0.69±0.11; AIC 185.6; BIC 194.8).
Nurses' BCMA-related perceptions explained 56%–76% of variance in nurses' BCMA acceptance (BI, SATISF). This finding is important because acceptance has been shown in other studies to predict the volume and nature of actual IT use as well as performance with IT.48 60 61 71 In fact, there is more evidence for the relationship between acceptance and IT use than there is for the relationship between IT system quality and use.72 The health IT literature abounds with case studies of unfavorable perceptions of health IT linked to non-use (overrides, workarounds, feature non-use), resistance, and even abandonment.36 73–79 There is also emerging quantitative evidence linking perceptions to health IT use.31 All of this promotes the need to properly measure, understand, and proactively shape clinical users' perceptions of health IT.
Nurses' perceptions of BCMA's ease of use (PEOU) and usefulness (PU, PU-PT) varied, with many nurses rating BCMA unfavorably on those dimensions. Variation in these ratings was associated with variation in BI and SATISF. Together with other recent BCMA research, this finding suggests that BCMA systems can be difficult to use for some and the costs of use may not be justified by BCMA's incremental benefit, resulting in a lack of acceptance, workarounds, and suboptimal use.21 24 25 28 Consequently, the improved usability of BCMA systems, achieved using appropriate usability methods,80–85 should be a top priority.86 Social influence from non-specific others (SI) and patients/families (SI-PT) were both rated highly and were significantly associated with BI and SATISF, respectively. One way to interpret this is that administrators, supervisors, and colleagues can influence nurses' BCMA use behavior (BI) but not their personal evaluation of BCMA (SATISF), whereas the reverse is true with patients and their families. More research is needed to understand specifically who influences nurses' and other clinicians' acceptance and use of health IT, by what means, and with how much success.87
As hypothesized, there was a benefit to adding variables to the base TAM, which was first developed in the mid-1980s based on the psychological Theory of Reasoned Action,88 and has been refined and expanded with new variables over the past quarter-century. However, social influence (SI, SI-PT) was the only added variable retained in the final models. Nevertheless, facilitating conditions (TRAINING, SUPPORT) and demographic variables (AGE, EXP) may still be important predictors in other contexts, as shown by recent technology acceptance studies with physicians using health IT such as electronic health records (EHRs).20 63 It is also possible that the impact of TRAINING and SUPPORT is indirect, mediated through PEOU and/or PU-PT. A third possibility is that the added variables used in this study were poorly contextualized and for that reason failed to predict acceptance: for example, informal ‘over-the-shoulder learning’ from colleagues may be a better predictor of acceptance than formal training.87
Our hypothesis about the benefit of contextualizing variables was largely confirmed in that the contextualized PU-PT was a better predictor of acceptance than the generic PU. The contextualized SI-PT was also a better predictor of SATISF than the generic SI. These findings motivate continued use of our contextualized measures of perceived usefulness and social influence. More work is needed to contextualize additional TAM variables such as PEOU and to identify additional contextualizations of PU and SI.
Overall, the proportions of variance explained in BI (56%) and SATISF (76%) were high and compare favorably with what is typically reported in the general TAM (30%–40%) and healthcare TAM (40%–70%) literatures.31 While BI is the standard acceptance measure in TAM, its use in mandatory settings bears further reflection. At the study hospital, BCMA use was expected by policy (indeed, nurses gave a mean response of 5.5 on the 0–6 scale in answer to the question, ‘To what extent is the decision to use the bar coding system beyond your control?’iii). The BI questions about anticipated future use yielded little variability, whereas responses about wanting to use the system in the future were more variable. Thus, nurses might have intended to use BCMA in the future without necessarily accepting it, making it important to measure SATISF as an indicator of acceptance.58 It is possible that nurses who are not satisfied with BCMA would report intending to use BCMA in the future but in actual practice might end up working around the system when they do not have to or might use BCMA only as much as needed to be compliant with policy and no more. Further work is needed to assess the relationship between BI, SATISF, and actual use or workarounds.
The results of the study may not generalize beyond the nurses at this particular hospital, and our results may not generalize beyond this particular vendor's BCMA. However, our results are generally consistent with a large body of TAM literature. Similarly, more important than generalization is the incremental development of theory, which can be tested across multiple settings. Given a <50% response rate and some missing values, findings may have been affected by survey and item non-response bias. Partly due to non-response, we suffered from having a sample size not large enough to permit modeling moderators or factors that predict perceptions. Some newly constructed scales were used but all had strong psychometric properties (table 1). The previously validated BI scale had less than desirable properties and appeared to mix predictions of behavior with motivation for behavior, as reflected in response distributions (table 3). As removing any one item from the BI scale resulted in worse internal consistency, we did not modify the scale, but urge improved conceptualization and measurement of BI in healthcare settings. We also may have failed to contextualize the variables appropriately and the contextualization of variables in this study may not apply to other clinicians, health IT, or healthcare settings. Our design was cross-sectional but the perceptions predicting acceptance may change over time,47 as one recent study of intensive care nurses' acceptance of EHRs found.89 Finally, we did not model actual use. This was intentional because use in this hospital was mandatory. However, future research on health IT in mandatory environments might consider modeling ‘effective use,’ ‘workarounds,’ or ‘depth of use’ as alternatives to typical use metrics such as ‘duration of use’ or ‘number of times used.’
Perceptions are more than the personal opinions or preferences of individuals: they are the building blocks of human behavior.88 90–92 By measuring nurses' perceptions related to BCMA, we were able to explain nurses' acceptance of BCMA, which is an antecedent to behavior. BCMA use behavior is of great importance in light of accumulating evidence that nurses override or work around BCMA even when policies are in place mandating BCMA use. This study, then, has important implications for health IT researchers and healthcare administrators/managers in charge of health IT implementation and improvement.
First, the findings support the need to contextualize models of technology acceptance or for that matter any other generic model or theory developed outside of healthcare. This is a key contribution as our results do not validate that the basic TAM model is best. Instead, in most cases a contextualized version is better. Specific methods such as belief elicitation interviews31 93 and literature-based methods94 are available to achieve contextualization. Contextualization also provides more specific strategies for designers, and the more contextualized, perhaps the better. Contextualization might need to address the target clinicians, the type of health IT, and the purpose for using the health IT. Our contextualization of PU to patient care shows that generically trying to improve nurse performance with BCMA will not be sufficient to improve acceptance; rather designers should focus on ensuring BCMA supports nurses' abilities to provide patient care. That is a different target than, for example, improving efficiency. An even more contextualized PU variable, perhaps one focused on medication management, might yield even more insight. Second, future research should attempt to improve on existing health IT TAM models in at least two ways: (1) by determining the predictors of PEOU and PU and (2) by determining the mediators of and moderators between predictors and acceptance. Variables that predict PEOU and PU should provide even more precise design guidance for developers. Mediators and moderators will be important for those in charge of managing implementations.
Modeling health IT acceptance should not be a solely academic pursuit. Results of empirical studies, such as the current one, have direct implications for actual implementations.40 In this case, the results suggest making BCMA easy to use, which is a recommendation for better design and usability testing, making BCMA useful for nursing work and patient care, and conveying to clinicians that key stakeholders promote their BCMA use for legitimate reasons.
The authors wish to thank the study participants and colleagues involved in the bar coding study. Two anonymous reviewers contributed helpful feedback.
Contributors: RJH had overall control of the design, analysis, and authoring of this study. The study was designed by RJH, with contributions from RB, MCS, and BTK, within the confines of the overall study originally conceptualized by BTK and MCS. RB was the biostatistician responsible for approving and executing analyses. RJH and BTK were responsible for interpreting study findings. All authors contributed to writing and gave final approval of the paper.
Funding: This work was supported by a grant from the Agency for Healthcare Research and Quality (AHRQ) (1 R01 HS013610) and the National Library of Medicine (1 R01 LM008923) to BTK. The funding agencies had no role in the study or publication except for providing funding for the study.
Competing interests: None.
Ethics approval: This study was approved by the University of Wisconsin-Madison Social and Behavioral Sciences IRB, and the IRB at the research hospital.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: Statistical analyses not reported in the paper (eg, standardized parameter estimates) both descriptive and analytic are available for direct distribution to anyone contacting the corresponding author. If desired, we will make additional statistical information available via supplementary or online materials or in additional tables, space permitting. Individual requests for additional analyses or raw data will be addressed on a case by case basis.
iSystem implementation was staggered but for practical purposes data collection was not. The post-BCMA time period is thus an average of the length of time that the study units were using BCMA at the time of the survey.
iiThe treatment of age and experience as direct predictors of acceptance differs from recent literature in which age, experience, gender, and voluntariness of use moderate the relationship between perceptions and acceptance. Gender and voluntariness of use were not modeled because there was very little variation on those variables in the sample.
iiiThis question was one of many asked on the full survey but not used in the present study's modeling analyses.