PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jamiaAlertsAuthor InstructionsSubmitAboutJAMIA - The Journal of the American Medical Informatics Association
 
J Am Med Inform Assoc. 2012 May-Jun; 19(3): 360–367.
Published online 2011 October 28. doi:  10.1136/amiajnl-2011-000289
PMCID: PMC3341782
Focus on health information technology, electronic health records and their financial impact

Medication administration quality and health information technology: a national study of US hospitals

Abstract

Objective

To determine whether the use of computerized physician order entry (CPOE) and electronic medication administration records (eMAR) is associated with better quality of medication administration at medium-to-large acute-care hospitals.

Data/study setting

A retrospective cross-sectional analysis of data from three sources: CPOE/eMAR usage from HIMSS Analytics (2010), medication quality scores from CMS Hospital Compare (2010), and hospital characteristics from CMS Acute Inpatient Prospective Payment System (2009). The analysis focused on 11 quality indicators (January–December 2009) at 2603 medium-to-large (≥100 beds), non-federal acute-care hospitals measuring proportion of eligible patients given (or prescribed) recommended medications for conditions, including acute myocardial infarction, heart failure, and pneumonia, and surgical care improvement. Using technology adoption by 2008 as reference, hospitals were coded: (1) eMAR-only adopters (n=986); (2) CPOE-only adopters (n=115); and (3) adopters of both technologies (n=804); with non-adopters of both technologies as reference group (n=698). Hospitals were also coded for duration of use in 2-year increments since technology adoption. Hospital characteristics, historical measure-specific patient volume, and propensity scores for technology adoption were used to control for confounding factors. The analysis was performed using a generalized linear model (logit link and binomial family).

Principal findings

Relative to non-adopters of both eMAR and CPOE, the odds of adherence to all measures (except one) were higher by 14–29% for eMAR-only hospitals and by 13–38% for hospitals with both technologies, translating to a marginal increase of 0.4–2.0 percentage points. Further, each additional 2 years of technology use was associated with 6–15% higher odds of compliance on all medication measures for eMAR-only hospitals and users of both technologies.

Conclusions

Implementation and duration of use of health information technologies are associated with improved adherence to medication guidelines at US hospitals. The benefits are evident for adoption of eMAR systems alone and in combination with CPOE.

Keywords: Health information technology, electronic medication administration record, computerized physician order entry, process quality, medication management, quality and cost of healthcare delivery, health information privacy and security

Introduction

Health information technologies (ITs) are at the center of the debate on improving quality and reducing cost in the US healthcare system. Federal incentives linked to the adoption of electronic health record (EHR) systems require organizations to demonstrate “meaningful use” of the technologies. Computerized physician order entry (CPOE) and electronic medication administration records (eMAR) are significant components of an EHR capable of meeting future meaningful use objectives.1 The Institute of Medicine has estimated that at least 1.5 million preventable adverse drug events occur annually in the USA, excluding errors of omission—failure to prescribe evidence-based medications that are likely to reduce morbidity and mortality.2 Advocates of health IT contend that the widespread use of systems such as CPOE and eMAR will improve the efficacy of care delivery and help with the challenges of medication management.3 However, recent studies have found inconsistent evidence supporting the association of EHR systems with quality and cost. In one such study, DesRoches and colleagues4 underlined the need to examine targeted use of EHR system components. Their results echo several studies, based on both national cross-sectional analysis and longitudinal analysis, suggesting that use of health ITs is only modestly associated with better quality.4–9

Over the past two decades, six systematic syntheses of prior work on the effect of health IT (including EHR systems, CPOE, clinical decision support systems, e-prescribing, telemedicine, and administrative functions) found some evidence of benefit.10–15 While, according to Buntin et al,10 there is growing evidence of a positive association between use of health IT and quality of care, particularly among studies using recent data, two recent meta-syntheses of review literature published during 1994–2010 (Black et al11 and Lau et al12) suggested that the gap between the postulated and empirically demonstrated benefits of health IT is still significant, and encouraged application of robust research methodologies to evaluate health IT against a comprehensive set of performance measures. Technologies such as CPOE in conjunction with clinical decision support systems have been shown to improve quality in specific organizational settings such as large academic hospitals that had developed in-house systems over long periods of time.6 7 13 14 16 17 However, critics argue that such contextual evidence may not be generalizable, because commercially available systems may vary significantly from technologies developed at pioneering institutions.8 For example, there is recent evidence of potential disadvantages in using systems such as CPOE that may lead to the introduction of unintended medication errors, potentially harming patients.18 Further, it is not simply technology alone but also experience in using the technology that matters for quality.18 19

From a policy perspective, such concerns are important given that the adoption and use of health IT in US hospitals is still quite varied. In 2009, Jha and colleagues reported that only 1.5% of US hospitals had what they term a “comprehensive” EHR system, with an additional 7.6% having a basic system.20 A subsequent survey in 2010 showed a marginal increase in EHR adoption—about 2.7% for comprehensive EHR and 7.9% for basic EHR system.21 Not surprisingly, small, public and rural hospitals are much less likely to have EHR systems. Moreover, 40% of US hospitals do not even have implementation plans for technologies such as CPOE.20 21 The uneven and non-random distribution of EHR systems in US hospitals means that simply looking at the relationship between presence, or better yet comprehensiveness, of EHR systems and quality can be problematic, even when controlling for hospital characteristics.16 22 In this retrospective observational study, our primary purpose is to examine the relationship between health IT use, measured by both presence of specific systems and the duration of their use, and adherence to medication guidelines for select conditions including acute myocardial infarction (AMI), heart failure (HF), pneumonia (PN), and surgical care infection prevention (SCIP). We focus on two core technologies supporting both ends of the medication management process: CPOE systems used for ordering medications, and eMAR systems used for administration of medications. In this study, we used a national sample of 2603 medium-to-large non-federal acute-care hospitals. We excluded small hospitals (<100 beds), which are least likely to have any IT systems. We also excluded use of pharmacy system technologies that support intermediary steps of the medication management process (eg, dispensing of medication) as they are already ubiquitous among medium-to-large hospitals (>99% had adopted). We analyzed the data using a generalized linear model with logit link to perform binomial logistic regression of medication quality measured during 2009 against health IT adoption status by 2008, controlling for several hospital characteristics. Regressing 2009 quality on IT adoption through 2008 ensured at least a 1-year lag between technology adoption and quality measurement, enabling us to tease apart the impact of technology use on hospital quality by comparing the adopters with non-adopters.

This study contributes to the literature in two important ways. First, as compared with earlier research, our results offer more consistent evidence for the association of specific health ITs (eMAR and CPOE) used in a targeted clinical area (ie, medication management process in acute-care setting) with a multitude of quality indicators related to the medication administration. Second, this study shows that the duration of technology use is associated with better performance on medication quality measures (learning by doing).

Methods

Design

We performed a retrospective cross-sectional analysis to determine whether the use of CPOE and eMAR by 2008 is associated with better quality of inpatient medication administration in 2009. The study used a national sample of 2603 non-federal acute-care hospitals to test two hypotheses with reference to 11 process quality measures. The quality measures, advanced by the Hospital Quality Alliance for inpatient settings, monitor utilization of medications for medical conditions including AMI, HF, and PN, as well as SCIP. Our primary hypothesis was that hospitals using eMAR and CPOE by 2008 would perform better on 2009 medication-related process quality measures compared with non-adopters. Recognizing that effective use of eMAR and CPOE depends on many factors, including organizational learning, simply examining the presence or absence of these technologies is not sufficient. Therefore we tested a second hypothesis to determine whether the duration of technology use by 2008 was associated with better performance in 2009.

We used a generalized linear model with logit as the linking function to perform binomial logistic regression. Analysis was conducted using Stata V.11.1. This specialized regression methodology is appropriate for the quality measures, as they are all proportions and bounded within zero and one. We conducted the analyses in two steps for 11 process quality measures as dependent variables measuring hospitals' adherence to medication-related quality for AMI, HF, PN, and SCIP. In the first step, to test the first hypothesis, we used three mutually exclusive indicator variables representing adopters of eMAR only, adopters of CPOE only, and adopters of both eMAR and CPOE. Our choice of these indicators, unlike the use of separate dummy variables for eMAR and CPOE in prior research,6 7 16 allowed us to differentiate hospitals in terms of their extent of technology in the medication management process as well capture the inherent staging of health IT adoption. In the second step, to test our second hypothesis, we used multivalued categorical variables representing duration of use (in 2-year multiples) by 2008 for adopters of eMAR only, adopters of CPOE only, and adopters of both eMAR and CPOE.

In each regression, non-adopters of both CPOE and eMAR were the reference group. To account for confounding effects, several hospital characteristics were included in the models: teaching status, profit status, rural location, qualification for disproportionate share payment, transfer-adjusted case mix index, bed size, and multihospital system membership. Additionally, regression errors were standardized at the hospital referral region level (representing the local market) to account for potential bias arising from any market level factors. Observational studies such as ours examining the relationship between hospital quality and health IT may suffer from endogeneity (selection bias),5 6 9 as high-quality hospitals may be more likely to adopt health IT.23 Past research has used several analytic strategies to overcome this methodological issue, such as panel data analysis, difference-in-difference approach, instrumental variable approach, and propensity-score-based adjustment.5 6 8 9 24 25 However, most of these studies report no significant difference in results after such adjustments, suggesting that endogeneity is not a significant concern. Yet, following this body of research, we leveraged a propensity-score-based adjustment to account for any potential estimation bias. First, we estimated IT adoption propensity scores using logistic regression analysis for health IT adoption against hospital characteristics as covariates. Subsequently, hospitals were classified using five indicators corresponding to the five quintiles of the propensity score distribution. We then included these indicators in each regression to control for endogeneity of IT use with quality.26 27 Finally, to test our second hypothesis, as several hospitals did not report adoption dates for eMAR and CPOE systems, we conducted the analysis on censored data—that is, by ignoring hospitals with missing observations on duration of use; and on imputed data, in which we derived duration of use by performing multiple imputations (n=25) using ordered logistic regression with select hospital characteristics as predictors to estimate the missing duration of use.28 Although the results based on the two approaches were qualitatively similar, we report the results from the censored data.

Setting

Data were drawn from three secondary sources. The Health Information and Management Systems Society (HIMSS) Analytics Database was licensed from HIMSS Analytics based on their survey of 5281 non-federal acute-care hospitals in the USA (December 2010 release update), which includes hospital characteristics and the operational status of clinical IT applications. In discussing advantages of HIMSS, McCullough (2008) noted, it “is the most comprehensive database of hospital IT adoption decisions” in the USA and has been available since the late 1980s.29 HIMSS follows an annual process to update the database, which involves initial data gathering conducted by phone followed by an IT inventory survey completed by hospital administrators. HIMSS provides benchmarking reports to respondents as an incentive for participation.29 The HIMSS Analytics data are used extensively by IT vendors and have been used widely in health services research.5–9 16 30–33 Some researchers have pointed to inconsistencies and low response rates to a different annual HIMSS survey of hospital chief information officers at about 200 hospital systems.20 We did not use these survey data in our analysis. In any given year, the HIMSS Analytics Database that we used represents nearly all of the 100+ bed non-federal hospitals (and >90% of all US hospitals).29

The Centers for Medicare and Medicaid Services (CMS) Hospital Compare Database provided a medication-related process quality measure for 3470+ non-federal acute-care hospitals collected during January–December 2009 (September 2010 release). To ensure a lag of at least 1 year between quality measurement and technology adoption, the adoption status for eMAR and CPOE in the 2010 HIMSS Database was censored at 2008. The CMS Acute Inpatient Prospective Payment System impact file (FY 2010 release) provided hospital characteristics data which were used as control variables.

Data collection

We constructed a national sample of acute-care hospitals by merging hospital level data from these three sources. We excluded all hospitals having fewer than 100 licensed beds, consistent with prior research.6 16 34 35 This allowed us to focus on hospitals most likely to invest in IT.20 21 Larger hospitals are also more likely to have the patient volume necessary for valid CMS quality measures.6 16 36

Measurement of IT for medication management

The HIMSS Database (December 2010 update) was used to code hospitals as users of eMAR and CPOE technologies with binary indicators. Hospitals were coded as technology users if they had reported these technologies to be in operational use by 2008 or before. Any hospital reporting these technologies to be in operational use after 2008 was recoded as a non-user. For analysis purposes, health IT use was operationalized through three mutually exclusive indicators: (1) eMAR-only adopters; (2) CPOE-only adopters; and (3) adopters of both technologies. The non-adopters of both CPOE and eMAR technologies were the reference group. Unlike prior research,37–41 which primarily focused on the effects of CPOE alone (or occasionally in combination with clinical decision support), this approach allowed us to better represent the extent of IT use in the ordering and administering phases of medication management.42 Although a typical medication management process at hospitals comprises ordering, dispensing, and administering of medications2 and is correspondingly supported by CPOE, a pharmacy information system, and eMAR, we ignored pharmacy information systems because they offer no meaningful variation (99.6% of the hospitals in our sample have such a system in operational use).

Additionally, to capture the effect of experience gained in using a system, hospitals were coded on an ordered scale (for each of the four technology categories) to represent duration of technology use in mutually exclusive 2-year increments: 0 for non-adopters; 1 for up to 2 years; and 2, 3, 4, and 5 for >2 to 4, >4 to 6, >6 to 8, and >8 years use, respectively. Subsequently, this variable was multiplied by the three technology adoption indicators to derive mutually exclusive, multivalued duration of technology use variables for eMAR-only adopters, CPOE-only adopters, and both eMAR and CPOE adopters, with non-adopters being the reference group.

Measurement of medication process quality

Our dependent variables were 11 evidence-based process measures for conditions including AMI, HF, and PN, as well as SCIP obtained from the CMS Hospital Compare Database (September 2010 release), which is based on data collected during January–December 2009. To ensure reliability of quality performance data, we ignored hospitals that reported quality scores for fewer than 25 eligible patients.6 43 44

Measurement of hospital characteristics

Lastly, the 2009 CMS Acute Inpatient Prospective Payment File was used for control variables on specific hospital characteristics. Consistent with prior research,4–6 44 we controlled for hospital size (natural log of number of licensed beds), transfer-adjusted case mix index (ratio), teaching status (1 if teaching, 0 otherwise), profit status (1 if for-profit, 0 otherwise), rural location (1 if rural, 0 otherwise), membership in a multihospital system (1 if yes, 0 otherwise), and whether a hospital qualified for Medicare disproportionate share adjustment (1 if qualified, 0 otherwise). In addition, we controlled for any hospital level learning effect with the natural log transform of condition-specific cumulative patient volume in the past 5 years—that is, 2004–2008.45–47

Results

Descriptive statistics

In our sample of 2603 hospitals, 30.9% (n=804) had both the eMAR and CPOE systems in operational use, 37.9% (n=986) had the eMAR system alone, 4.4% (n=115) had the CPOE system alone, and 26.8% (n=698) had neither technology by 2008 (table 1). The overall adoption rate for eMAR and CPOE systems was 68.8% (n=1790) and 35.3% (n=919), respectively. Among the eMAR and CPOE users, about 44% (n=788) and 39% (n=357) hospitals, respectively, did not report the year of technology adoption. Using historical data releases from HIMSS for the 1990s up to 2007, we were able to determine the year of adoption for some missing observations (128 hospitals for eMAR and 68 hospitals for CPOE) leading to about 33.4% (n=598) and 27.3% (n=251) hospitals, respectively, with missing adoption years for eMAR and CPOE. Among hospitals reporting the adoption year, the majority adopted eMAR and CPOE systems during 2003–2004 (eMAR, 27%; CPOE, 26%) and 2005–2006 (eMAR, 16.9%; CPOE, 17.0%). To verify differences in quality performance between hospitals reporting technology adoption year and those not reporting, we conducted a two-tailed t test and found no systematic differences (see table A1 in the online appendix).

Table 1
Adoption and duration of use of eMAR and CPOE technologies in medium-to-large acute-care hospitals in 2008 (N=2603)

Table 2 summarizes the characteristics of all acute-care hospitals in the sample. Among the 2603 hospitals, 37.5% were teaching, 19.1% were for-profit, 72.2% were affiliated with a multihospital health system, 16.2% were located in rural areas, and 81.1% qualified for disproportionate share payment. Finally, the majority of hospitals had between 100 and 299 beds (100–199 beds, 39%; 200–299 beds, 24%), and the mean transfer-adjusted case mix index was 1.49 (SD=0.24). Hospitals with eMAR systems (n=1790), on the one hand, were similar to the overall sample of hospitals on most characteristics, except that eMAR adopters had a higher transfer-adjusted case mix, varied by bed size, and were less likely to qualify for disproportionate share payments. On the other hand, CPOE adopters were more likely to be teaching, non-profit, urban hospitals, and had higher transfer-adjusted case mix.

Table 2
Characteristics of medium-to-large acute-care hospitals (n=2603)*

Not all of the 2603 hospitals reported on all 11 quality measures (table 3). The number of hospitals reporting quality measures varied from a low of 1071 (for AMI3: eligible AMI patients given angiotensin converting enzyme (ACE) inhibitor or angiotensin receptor blockers (ARB) for left ventricular systolic dysfunction) to a high of 2296 (for PN5: eligible PN patients given initial antibiotics within 6 h of arrival). The overall sample averages for reporting hospitals on each quality measure during January–December 2009 ranged from 92.17% to 98.71%. For example, on average 98.71% (SD=1.61%) of eligible AMI patients (with no known contraindications) were given aspirin upon arrival, whereas 92.19% (SD=6.39%) of eligible surgery patients received treatment to prevent blood clots within 24 h before or after selected surgeries. The average quality scores for reporting hospitals with eMAR systems ranged over 92.57–98.82%, and without eMAR systems ranged over 90.95–98.40%. Likewise, the average quality scores among reporting hospitals with CPOE systems ranged over 92.83–98.85%, and without CPOE systems ranged 91.58–98.64%. While the quality scores for our sample averaged in the 90s, hospital level scores were distributed over large ranges: nine of 11 measures had minimum scores from low 10s to 50s, and the remaining two measures had minimum scores of 67% and 76% (for brevity, the range of quality scores is not shown in table 3). For example, in the case of PN, the proportion of eligible patients given the most appropriate initial antibiotics varied from 10% to 100% among 2279 reporting hospitals, and in the case of AMI, the proportion of eligible patients given ACE inhibitor or ARB for left ventricular systolic dysfunction varied from 67% to 100% among 1071 reporting hospitals. These data show that the adherence to medication-related guidelines not only varies substantially across hospitals, but also the number of hospitals reporting quality of care data varies across measures.

Table 3
Summary statistics of observed medication process quality at medium-to-large acute-care hospitals in 2009 (n=2603)

Association of eMAR and CPOE use with medication process quality

Our results are summarized in figure 1, which shows the adjusted ORs (AORs) and 95% CIs for all the measures in each group (eMAR-only, CPOE-only, and with both technologies) relative to non-adopters of both technologies (reference group). We found that hospitals that implemented eMAR only performed better on 10 of 11 process quality measures, relative to non-adopters of both eMAR and CPOE. For these hospitals, the odds of adhering to recommended medication guidelines were 14–29% higher than for non-adopters. For example, the odds of giving aspirin to AMI patients at admission were 29% (AOR 1.29; 95% CI 1.11 to 1.48) higher, whereas the odds of administering preventive antibiotic(s) within 1 h before incision for surgical patients were 14% (AOR 1.14; 95% CI 0.98 to 1.34) higher among adopters of eMAR-only systems compared with non-adopters. Only the measure tracking the prescription of β blockers at discharge to AMI patients showed no statistically significant difference between the two groups of hospitals.

Figure 1
Adjusted ORs (95% CIs) for electronic medication administration records (eMAR) and computerized physician order entry (CPOE) adoption on medication process quality at medium-to-large acute-care hospitals in the USA (n=2603). AMI1: given aspirin at admission; ...

In contrast, hospitals that implemented CPOE only showed little difference in quality from non-adopters. In only two of the 11 measures did CPOE-only adopters have higher quality than non-adopters: the odds of giving the most appropriate initial antibiotic(s) to PN patients (AOR, 1.29; 95% CI 1.08 to 1.54), and giving treatment to prevent blood clots within 24 h before or after select surgery to eligible patients (AOR, 1.27; 95% CI 0.99 to 1.63; significant at p<0.10).

Hospitals that adopted both eMAR and CPOE performed better on 10 of the 11 measures. The odds of adherence to medication guidelines were 13–38% higher among adopters of both technologies compared with non-adopters. For example, the odds of prescribing aspirin at discharge were 38% (AOR, 1.38; 95% CI 1.38 to 1.68) higher, whereas the odds of administering preventive antibiotic(s) within 1 h before incision to surgical patients were 13% (AOR, 1.13; 95% CI 1.00 to 1.29) higher among hospitals using both eMAR and CPOE compared with non-adopters.

In all, these results show substantive support for hypothesis 1, with the exception of those hospitals that adopted CPOE only. We will explore this issue further in the comments section.

Association of duration of eMAR and CPOE use with medication process quality

In our sample, a substantial number of hospitals did not report adoption dates for eMAR and CPOE technologies (33% and 27%, respectively, among adopters). To examine the relationship between duration of use (measured in 2-year increments) of health IT and hospital quality (hypothesis 2), we conducted the analysis in two ways. First, we performed the analysis with a censored dataset—that is, by ignoring hospitals with missing observations on duration of use. Next we repeated the analysis using imputed data, in which we derived duration of use by performing multiple imputations (n=25) using ordered logistic regression, with select hospital characteristics as predictors to estimate the missing duration of use. The results from the two approaches were qualitatively similar. Here we report the results using censored data.

Figure 2 provides a summary of the AORs and 95% CIs for duration of eMAR and CPOE use on the medication-related quality measures (see table A2 in the online appendix for the results based on multiple imputations). For each additional 2 years of use among hospitals with eMAR-only systems, the odds of adhering to the recommended medication guidelines were significantly higher on all 11 measures, with gains of 6–15% compared with hospitals with neither eMAR nor CPOE systems. For experience with CPOE-only systems, however, we found no difference in quality between such hospitals and non-adopters of both eMAR and CPOE systems on most measures; only the odds of prescribing β blocker to AMI patients (AOR, 1.14; 95% CI 1.02 to 1.28), giving most appropriate initial antibiotic(s) to PN patients (AOR, 1.08; 95% CI 1.01 to 1.16), and giving appropriate medication to prevent blood clots within 24 h before or after surgery to surgical patients (AOR, 1.10; 95% CI 1.01 to 1.19) were higher. In contrast, for each additional 2 years of use of both eMAR and CPOE systems, the odds of adhering to the recommended medication guidelines were 9–15% higher across nine of the 11 quality measures. These results suggest that duration of technology use is associated with better quality of medication administration (supporting hypothesis 2).

Figure 2
Adjusted ORs (95% CIs) for duration of electronic medication administration records (eMAR) and computerized physician order entry (CPOE) use, measured in the increment of 2 years, on medication process quality at medium-to-large acute-care hospitals ...

Association of hospital characteristics with medication process quality

In our sample, teaching status, profit status, and rural hospitals had generally no significant association with medication process quality measures, except that rural hospitals were likely to perform better on one PN and one surgical infection prevention measure. Further, membership to multihospital systems was positively associated with most medication quality measures; however, hospital bed size, case mix index, qualification for disproportionate share payment, and cumulative patient volume were negatively associated. Finally, technology adoption propensity was mostly non-significant (see tables A3 and A4 in the online appendix for details).

Comments

This study sought to quantify the association between the use of eMAR and CPOE with the quality of inpatient medication administration. In particular, it compared hospitals using eMAR only, CPOE only, and both eMAR and CPOE against hospitals not using either technology, as well as duration of use of those technologies, with respect to their adherence to 11 medication-related clinical guidelines. The results show higher adherence to 10 of 11 measures for hospitals using both eMAR and CPOE (AORs ranging from 13% to 38%) as well as hospitals using eMAR only (AORs ranging from 14% to 29%). Similar results hold for duration of technology use, indicating that hospital experience with the technologies is important (AORs ranged from 6% to 15% for eMAR-only hospitals and users of both eMAR and CPOE). These AORs translate to about 0.4 to 1.4 percentage point improvements in the adherence to medication guidelines among hospitals using eMAR only, and about 0.4 to 2.0 percentage point improvements among hospitals using both eMAR and CPOE systems compared with non-adopters of both technologies (see table A5 in the online appendix for details of marginal effects of eMAR and CPOE). These marginal improvements associated with use of health IT are substantive and consistent with prior research findings reported by DesRoches et al,4 Jones et al,5 and McCullough et al,7 despite the “ceiling effects”—that is, the average quality scores on all quality measures are in the 90s.5

Prior studies of health IT and quality, examining national samples of hospitals, have reported mixed findings.6 16 48 49 For example, studies that focused on full-scale EHR systems or combinations of several technologies have found limited evidence for improved medication quality.8 9 42 48 49 Two studies found that hospitals with CPOE outperform hospitals without CPOE on AMI and PN medication measures reported for 2004.6 16 In both studies, however, CPOE adoption was negatively associated with administration of initial antibiotics within 4 h of arrival, a measure also included in our study (note that the time threshold was extended by Hospital Quality Alliance to 6 h in 2007). Our findings suggest that hospitals with CPOE perform better on this measure, but only when the hospitals also have eMAR (AOR=1.19, 95% CI 1.04 to 1.35). In CPOE-only hospitals, there was no significant difference on this measure compared with non-adopting hospitals. The positive result on this measure in our analysis may be due to several factors including the concurrent use of eMAR systems in the medication management process, the increase in the time threshold from 4 to 6 h in the guidelines, our use of more recent data, or our focus on 100+ bed hospitals.

In summary, this study showed positive evidence that use of, and experience with, eMAR technology, alone and in combination with CPOE, is associated with a better quality of medication administration compared with hospitals without such technologies. The effects of using eMAR in combination with CPOE, however, do not appear to be significantly different from the effects of eMAR alone. This lack of marginal improvement with the addition of CPOE is surprising, but may be due to several confounding factors not included in this study, such as variations in functional capabilities of CPOE systems, usability factors, and variations in the extent of technology use among clinical staff. Such clarity over capabilities and extent of use is not reflected in our data.50 As David Bates51 noted “A CPOE system can have a major effect on care improvement, but realizing its potential benefits will demand that it be used well.” This observation is supported by many implementation challenges in the clinical decision support module, a key component in CPOE.52

This study has a number of limitations. Cross-sectional analyses such as those reported in this study do not represent causal effects of IT use on process quality. We examined IT use as of 2008 on quality in 2009, and employed duration of technology use to try to determine the relationship between quality and effects of technology use over time and find a significant positive relationship. However, future studies are needed that use panel datasets to tease out the causal relationship. A second limitation is that the data for a substantial number of hospitals were missing technology adoption year, so could not be included in the analysis of duration of technology use. We found no significant differences in the quality scores between reporting and non-reporting hospitals, suggesting that non-reporting hospitals do not differ systematically from reporting hospitals. Still, we imputed duration of use from hospital characteristics in order to include missing hospitals in our analysis. Third, our results focus on acute-care hospitals with 100 or more beds, so they do not necessarily apply to small hospitals. Fourth, we cannot rule out the possibility that hospitals with health IT systems, such as those studied here, were merely better at recording their medication administration than non-adopting hospitals. Finally, our measures of eMAR and CPOE did not capture aspects of functionality or other important dimensions of technology use, such as extent of use among clinicians, that may influence the quality of medication administration. Despite these limitations, the findings of this study show a positive relationship between specific types of health IT use and quality of medication administration in medium and large US hospitals.

Conclusions

Acute-care hospitals that adopted eMAR and CPOE systems before 2008 performed better on 2009 medication quality measures related to conditions including AMI, HF, and PN, as well as preventive medications for surgery patients. Adoption of eMAR alone, and in combination with CPOE, is associated with greater adherence to medication guidelines. These findings, compared with past research, offer more consistent and stronger evidence for the benefits of health IT. Additionally, duration of use for these technologies (eMAR alone, and the combination of eMAR and CPOE) is also associated with better adherence, thus plausibly demonstrating the effects of “learning by doing”. However, hospitals using CPOE alone (about 5% of the sample) did not perform better than non-adopters of eMAR and CPOE. Further work is needed to understand the role of the specific functionality of eMAR and CPOE systems in medication quality, such as the variation in capabilities of these systems, the extent of technology use among clinical staff, as well as how health IT fits with specific organizational and practice characteristics that facilitate the delivery of high-quality care.

Supplementary Material

Supplementary Data:

Acknowledgments

We thank HIMSS Analytics for licensing data on hospitals' adoption of eMAR and CPOE. We thank Dr Ohno-Machado (editor-in-chief), associate editor, and anonymous reviewers of JAMIA for their valuable suggestions. Additionally, we thank Academy Health conference attendees, and seminar participants at the School of Informatics, Indiana University Purdue University at Indianapolis for their helpful comments.

Footnotes

Funding: National Science Foundation (NSF), Grant Number NSF-CNS-0910842; Rockefeller Center for Public Policy at Dartmouth College. Neither National Science Foundation nor Rockefeller Center for Public Policy at Dartmouth College had any role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, or the approval of manuscript.

Competing interests: None.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Classen DC, Bates DW, Denham CR. Meaningful use of computerized prescriber order entry. J Patient Saf 2010;6:15–24. [PubMed]
2. Institute of Medicine Preventing medication errors. In: Aspden P, Wolcott JA, Bootman JL, et al., editors. , eds. Committee on Identifying and Preventing Medication Errors, Board of Health Care Services. Washington DC: National Academy Press, 2007.
3. Blumenthal D, Glaser JP. Information technology comes to medicine. N Engl J Med 2007;356:2527–34. [PubMed]
4. DesRoches CM, Campbell EG, Vogeli C, et al. Electronic health records' limited successes suggest more targeted uses. Health Aff (Millwood) 2010;29:639–46. [PubMed]
5. Jones SS, Adams JL, Schneider EC, et al. Electronic health records adoption and quality improvement in US Hospitals. Am J Manag Care 2010;16:SP64–71. [PubMed]
6. Kazley AS, Diana ML. Hospital computerized provider order entry adoption and quality: an examination of the United States. Health Care Manage Rev 2011;36:86–94. [PubMed]
7. McCullough JS, Casey M, Moscovice I, et al. The effect of health information technology on quality in U.S. hospitals. Health Aff (Millwood) 2010;29:647–54. [PubMed]
8. Himmelstein DU, Wright A, Wooldhandler S. Hospital computing and the costs and quality of care: a national study. Am J Med 2010;123:40–6. [PubMed]
9. Kazley AS, Ozcan YA. Do hospitals with electronic medical records provide higher quality care? An examination of three clinical conditions. Med Care Res and Rev 2008;65:496–513. [PubMed]
10. Buntin MB, Burke MF, Hoaglin MC, et al. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood) 2011;30:464–71. [PubMed]
11. Black AD, Car J, Pagliary C, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011;8:e1000387. [PMC free article] [PubMed]
12. Lau F, Kuziemsky C, Price M, et al. A review on systematic reviews of health information studies. J Am Med Inform Assoc 2010;17:637–45. [PMC free article] [PubMed]
13. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742–52. [PubMed]
14. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA 2005;293:1261–3. [PubMed]
15. Goldzweig CL, Towfigh A, Maglione M, et al. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood) 2009;28:w282–93. [PubMed]
16. Yu FB, Menachemi N, Berner ES, et al. Full implementation of computerized physician order entry and medication-related quality outcomes: a study of 3364 hospitals. Am J Med Qual 2009;24:278–86. [PubMed]
17. Zlabek JA, Wickus JW, Mathiason MA. Early cost and safety benefits of an inpatient electronic health record. J Am Med Inform Assoc 2011;18:169–72. [PMC free article] [PubMed]
18. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293:1197–203. [PubMed]
19. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc 2007;14:542–9. [PMC free article] [PubMed]
20. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in US hospitals. N Engl J Med 2009;360:1628–38. [PubMed]
21. Jha AK, DesRoches CM, Kralovec P, et al. A progress report on electronic health records in U.S. hospitals. Health Aff (Millwood) 2010;29:w1–7. [PubMed]
22. Bates DW. The effects of health information technology on inpatient care. Arch Intern Med 2009;169:105–7. [PubMed]
23. Elnahal SM, Joynt KE, Bristol SJ, et al. Electronic health record functions differ between best and worst hospitals. Am J Manag Care 2011;17:e121–47. [PMC free article] [PubMed]
24. Furukawa MF. Electronic medical records and the efficiency of hospital emergency departments. Med Care Res Rev 2011;68:75–95. [PubMed]
25. Miller AR, Tucker CE. Encryption and the loss of patient data. J Policy Anal Manage 2011;30:534–56. [PubMed]
26. Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757–63. [PubMed]
27. Rosenbaum PR, Rubin DB. Reducing bias in observational studies using sub-classification on the propensity score. J Am Stat Assoc 1984;79:516–24.
28. Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev Sci 2007;8:206–13. [PubMed]
29. McCullough JS. The adoption of hospital information systems. Health Econ 2008;17:649–64. [PubMed]
30. Parente ST, Van Horn L. Hospital investment in information technology: does governance makes a difference? Health Care Financ Rev 2007;28:31–43. [PMC free article] [PubMed]
31. Wang BB, Wan TT, Burke DE, et al. Factors influencing health information system adoption in American hospitals. Health Care Manage Rev 2005;30:44–51. [PubMed]
32. Furukawa MF, Raghu TS, Spaulding TJ, et al. Adoption of health information technology for patient safety in US hospitals 2006. Health Aff (Millwood) 2008;27:865–75. [PubMed]
33. Fonkych K, Taylor R. The state and pattern of health information technology adoption. Santa Monica, Calif: RAND Health, 2005. http://www.rand.org/pubs/monographs/2005/RAND_MG409.pdf.
34. Lehrman WG, Elliott MN, Goldstein E, et al. Characteristics of hospitals demonstrating superior performance in patient experience and clinical process measures of care. Med Care 2010;67:38–55. [PubMed]
35. Kazley AS, Ozcan YA. Electronic medical record use and efficiency: A DEA and window analysis of hospitals. Socio-Econ Plan Sci 2009;43:209–16.
36. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume-outcome relationship. Health Serv Res 2010;45:1148–67. [PMC free article] [PubMed]
37. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280:1311–16. [PubMed]
38. Cunningham TR, Geller ES, Clarke SW. Impact of electronic prescribing in hospital setting: a process-focused evaluation. Int J Med Inform 2008;77:546–54. [PubMed]
39. Shamliyan TA, Duval S, Du J, et al. Just what the doctor ordered: review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2008;43:32–53. [PMC free article] [PubMed]
40. Van Rosse F, Maat B, Rademaker CM, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009;123:1184–90. [PubMed]
41. Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients: a systematic review. Int J Med Inform 2008;77:365–76. [PubMed]
42. Furukawa MF, Raghu TS, Shao BB. Electronic medical records, nurse staffing, and nurse-sensitive patient outcomes: evidence from California hospitals 1998-2007. Health Serv Res 2010;45:941–62. [PMC free article] [PubMed]
43. Shwartz M, Ren ZJ, Pekoz E, et al. Estimating a composite measure of hospital quality from the hospital compare database. Med Care 2008;46:778–85. [PubMed]
44. Jha AK, Li Z, Orav EJ, et al. Care in U.S. hospitals–The hospital quality alliance program. N Eng J Med 2005;353:265–74. [PubMed]
45. Theokary C, Ren ZJ. An empirical study of the relations between hospital volume, teaching status and service quality. Prod Oper Man 2011;20:303–18.
46. Joynt KE, Orav J, Jha AK. The association between hospital volume and processes, outcomes, and cost of care for congestive heart failure. Ann Intern Med 2011;154:94–102. [PMC free article] [PubMed]
47. Birkmeyer JD, Finlayson EV, Birkmeyer CM. Volume standards for high-risk surgical procedures: potential benefits of the Leapfrog initiative. Surgery 2001;130:415–22. [PubMed]
48. Jha AK, Orav J, Ridgway AB, et al. Does the Leapfrog Program help identify high-quality hospitals? Jt Comm J Qual Patient Saf 2008;34:316–25. [PubMed]
49. Yu F, Houston TK. Do ”most wired“ hospitals deliver better care? Jt Comm J Qual Patient Saf 2007;33:136–44. [PubMed]
50. Diana ML, Kazley AS, Menachemi N. An assessment of Health Care Information and Management Systems Society and Leapfrog data on computerized provider order entry. Health Serv Res 2011;46:1575–91. [PMC free article] [PubMed]
51. Bates DW. CPOE and clinical decision support in hospitals: Getting benefits: Comment on “Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction”. Arch Intern Med 2010;170:1583–4. [PubMed]
52. Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14:29–40. [PMC free article] [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association