Electronic Health Record Functions Differ Between Best and Worst Hospitals

Am J Manag Care. Author manuscript; available in PMC 2012 April 24.

Published in final edited form as:

Am J Manag Care. 2011 April; 17(4): e121–e147.

PMCID: PMC3335431

NIHMSID: NIHMS368522

Electronic Health Record Functions Differ Between Best and Worst Hospitals

Shereef M. Elnahal, BA, Karen E. Joynt, MD, Steffanie J. Bristol, BS, and Ashish K. Jha, MD, MPH

Harvard Medical School (SME), Boston, MA; Department of Health Policy and Management, Harvard School of Public Health (KEJ, SJB, AKJ), Boston, MA; Division of Cardiovascular Medicine (KEJ), Brigham and Women’s Hospital, Boston, MA; General Internal Medicine (AKJ), Brigham and Women’s Hospital, Boston, MA; VA Boston Healthcare System (AKJ), Boston, MA

The publisher's final edited version of this article is available free at Am J Manag Care

See other articles in PMC that cite the published article.

Abstract

Objectives

To determine if patterns of Electronic Health Records (EHRs) adoption and Meaningful Use vary between high, intermediate, and low quality U.S. hospitals.

Study Design

We used data from the Hospital Quality Alliance (HQA) program to designate hospitals as high quality (performance in the top decile nationally), low quality (bottom decile) and intermediate quality (all others). We examined EHR adoption and Meaningful Use using national survey data.

Methods

We used logistic regression models to determine the frequency with which hospitals in each group adopted individual EHR functions and met Meaningful Use criteria, and factor analyses to examine patterns of adoption in high and low quality hospitals.

Results

High quality hospitals were more likely to have all clinical decision support functions. High quality hospitals were also more likely to have computerized physician order entry for medications compared to intermediate and low quality hospitals. Among those who had not yet implemented components of clinical decision support, two-thirds of low quality hospitals reported no concrete plans for adoption. Finally, high quality hospitals were more likely to meet many of the Meaningful Use criteria, such as reporting quality measures, implementing at least one clinical decision support rule, and exchanging key clinical data.

Conclusions

We found higher rates of adoption of key EHR functions among high quality hospitals, suggesting that high quality and EHR adoption may be linked. Most low quality hospitals without EHR functions reported no plans to implement them, pointing to challenges faced by policymakers in achieving widespread EHR adoption while simultaneously improving quality of care.

Introduction

The U.S. has embarked on an ambitious effort to promote the adoption and Meaningful Use of electronic health records (EHRs) and the key functionalities that underlie these systems.^{1, 2} The motivation for this effort is simple: the current system of paper-based records exacerbates deficiencies in information and can lead to piecemeal, poor quality care. Electronic health records, when properly designed and implemented, can provide more complete, timely, and sophisticated clinical information and support to clinicians, and therefore improve the quality of care delivered to patients.^3–6 There has been broad, bi-partisan interest in EHRs, initially with the Bush Administration and now, in the Obama Administration. Most recently, the American Recovery and Reinvestment Act (ARRA) allocated nearly $30 billion in direct incentives designed to encourage physicians and hospitals to adopt and use these systems through “Meaningful Use”.⁷

Since the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, several studies have called into question the relationship between EHR use and quality of care.^{8, 9} These data have fueled criticisms of current efforts to promote EHR adoption; skeptics point to these studies to argue that there is inadequate evidence to support widespread EHR use. However, studies demonstrating only modest overall effects of EHRs on quality of care may miss important differences in EHR use between the best and worst hospitals. If the underlying goal is to improve quality, examining how high quality hospitals in the U.S. use EHRs, and determining whether this is substantively different than how poor- quality hospitals use EHRs, could provide important insights for clinicians and policymakers seeking to move providers towards the provision of higher quality care. Further, understanding which specific EHR functionalities are in use among the high quality hospitals could provide guidance in terms of how low or intermediate quality hospitals might focus their EHR efforts going forward.

Therefore, we used national data on patterns of EHR adoption to address four key questions. First, are there differences in the adoption of specific EHR functionalities, such as medication lists, computerized prescribing, or clinical decision support, between high and low quality hospitals? Second, if these differences exist, which functionalities display the largest disparities in adoption when comparing high and low quality hospitals? Third, do the highest quality hospitals seem to have different patterns of adoption than the lowest quality hospitals (i.e. do the cluster of functions adopted vary between the high and low quality institutions)? Fourth, among those hospitals which have not yet adopted individual functionalities, are there important differences between high and low quality hospitals in their current plans to implement them? And finally, are there differences in adoption of the specific functions that comprise the newly established Meaningful Use Criteria¹⁰ for Electronic Health Records adoption?

Methods

Measures of Electronic Health Record Functions

We used two primary data sources for this analysis: the 2009 American Hospital Association (AHA) hospital IT survey of US acute care hospitals and the 2006 the Hospital Quality Alliance database. The AHA IT survey was distributed as a supplement to the AHA’s annual survey in 2009. This has served as a data source for many analyses and the details of its development and distribution are described in prior publications.¹⁰ The survey was adminstered to all 4,493 acute care hospitals in the AHA (an estimated 97 percent of all hospitals in the U.S.) from March–September 2009. Completed surveys totaled 3,101, for a 69 percent response rate. The survey assessed the level of adoption of specific EHR functionalities. Respondents were asked to report a score of one through six to assess the degree of adoption for each functionality, ranging from full adoption of the function across all units to a declaration that the functionality was not in place and that there were no plans or considerations to implement it. We focused on the 24 electronic functions that a federally-sanctioned expert panel identified as part of a comprehensive EHR.¹⁰

Measures of Quality

We used data from the Hospital Quality Alliance, which contains information on process measures for patients cared for during calendar year 2006. We created summary scores for performance on care for acute myocardial infarction (AMI), congestive heart failure (CHF), pneumonia, and prevention of surgical complications.¹¹ The specific indicators are summarized in the appendix (Appendix Table 6). We took an average of each hospital’s summary score within each of the four clinical areas and ranked all the hospitals in order of performance. We excluded hospitals with fewer than 30 observations for any of the four clinical conditions of interest, as well as hospitals located outside of the fifty states or the District of Columbia.

Appendix Table 6

Hospital Quality Alliance Quality of Care Process Measures

Analysis

We began by categorizing the hospitals in our sample into quality deciles based on their overall quality score and created three groups for our main analysis: hospitals in the top ten percent of performance were designated as high quality, those in the bottom ten percent were designated as low quality and all other hospitals (those in deciles 2 through 9) were designated as intermediate quality. In sensitivity analyses, we examined other cut-points for designating hospitals as high versus low quality, including the top and bottom 20% as well as top and bottom 30%. We calculated the proportion of hospitals within each cohort (high quality, medium quality, and low quality) that had adopted each EHR functionality in at least one hospital unit. We used chi squared tests to compare the proportions of hospitals that had adopted each function across the three groups. To account for potential confounding, we built multivariate logistic regression models, adjusting for hospital size, region, ownership (for-profit, non-profit, or public), teaching status, membership in a hospital system, urban vs. nonurban location, the presence of a cardiac intensive care unit (an indicator of technological capacity), and the percentage of each hospital’s patients who were covered by Medicaid (an indicator of the socioeconomic status of patients treated in each hospital). For each specific functionality, hospitals with missing data were excluded from that calculation. We only included the presence of several key decision support tools related to medication alerts if the hospital also had computerized provider order entry (CPOE) for medications. This was done to reflect true decision support at the point of care by health care providers, which would require the presence of electronic order entry. We reran our analyses without the requirement for CPOE and our results were qualitatively similar. Thus, only present the findings of those decision support tools in the presence of CPOE.

Next, we used factor analysis to determine the covariance of adoption of functionalities within each of the quality cohorts. We simply describe the patterns of clustering of functions across the three quality cohorts.

Using the same groups but limiting our analysis this time to those hospitals which had yet to implement each EHR functionality, we calculated the proportion of hospitals that reported no concrete plans for implementation. This was defined as the proportion reporting that they had considered implementing but had no resources identified for implementation or that they had no plans to implement. We compared the frequency of these responses across the three groups initially using chi-squared tests and subsequently, using multivariate logistic regression analyses as described above to adjust for potential confounders.

Finally, we examined the proportion of hospitals within each quality cohort that had adopted the specific functions required to meet Meaningful Use criteria. This included twelve objectives that had clear analogues to the AHA health IT survey (nine of the fourteen Core Objectives and three of the ten Menu Objectives; Appendix Table 7). For these analyses, we used chi-squared tests to determine if the proportion of adopters varied across these three groups and did not exclude missing data from calculations.

Appendix Exhibit 7

Final Stage 1 Meaningful Use Rule for Eligible Hospitals and Analogous AHA IT Survey Question

There were slight differences between hospitals that did and did not respond to the health information technology survey.¹⁰ In the analyses reported, all results were weighted to account for the differences due to nonresponse using a previously described method.¹⁰. All analyses were performed using Stata/SE, Version 10.1, College Station, TX. A two-sided p-value less than 0.05 was considered to be statistically significant.

Results

Of the 1,637 hospitals in our sample, 166 were designated as high quality, 1318 as intermediate quality, and 153 as low quality (Table 1). There were substantial differences in the characteristics of these hospitals: high quality hospitals were more often large compared with low quality hospitals (26% versus 8%, p<0.001), and more often non-profit in ownership (84% versus 49%, p<0.001). High quality hospitals were significantly more likely to be teaching hospitals than low quality hospitals (44% versus 23%, p<0.001), belong to a hospital system (71% versus 55%, p <0.005), located in urban areas (86% versus 59%, p<0.001), and have a dedicated coronary intensive care units (62% versus 28%, p<0.001). Finally, the percentage of patients with Medicaid was substantially lower in the high quality than the low quality hospital cohort (9 % versus 15%, p<0.001).

Table 1

Baseline Characteristics, by Quality Cohort, among responders to the AHA IT Survey

We found substantial differences in the adoption of EHR functions among the three groups of hospitals (Table 2). High quality hospitals more often had electronic nursing notes (81% versus 73% and 68%, p = 0.04) and medication lists (89% versus 79% and 73%, p < 0.01) than intermediate and low quality hospitals, respectively. All “decision support” tools had significantly higher adoption levels in the high quality cohort. The differences between the high and low quality cohorts in adoption of all of these functions ranged from 17% to 20%, and all were significant (Table 2).

Table 2

Proportion of hospitals with selected electronic functionalities implemented in at least one unit in the high, intermediate, and low quality grades

After multivariable adjustment, we found that adoption of 22 of the 24 functions was still higher in high quality hospitals, although most of the differences were no longer statistically significant (Appendix Table 1). Functions for which the differences across the three quality cohorts were statistically significant included problem lists, medication lists, diagnostic test images, and many of the clinical decision support tools.

Appendix Table 1

Multivariable-adjusted Proportions and Differences of Proportions of Hospitals with Selected Electronic Functionalities Implemented in At Least One Unit in the Highest, Intermediate, and Lowest Quality Groups

In sensitivity analyses, when we examined groupings based on alternative cutpoints, we found that most of the results were qualitatively similar. However, expanding the high and low quality groups to the 30% cutoff decreased the differences between groups, some of which became nonsignificant (see Appendix).

We performed separate factor analyses in each of the three cohorts of hospitals (high quality, intermediate quality, and low quality) and found relatively similar results across all three groups. Within each cohort, there were two factors with relatively high Eigen values (greater than 3). For example, among the high quality cohort, the hospitals differed most in terms of whether they had adopted CPOE and decision support. The second factor clustered together adoption of patient demographics with viewing lab and radiology reports. The patterns were very similar in the intermediate and low quality cohorts (see Appendix Tables 3a–c).

Appendix Table 3a

Factor Loadings for High Quality Hospitals

Appendix Table 3c

Factor Loadings for Low Quality Hospitals

Among those hospitals which had yet to implement specific EHR functions, we found high rates of hospitals reporting that they had no concrete plans to implement many key functionalities (Table 3). For clinical documentation, results viewing, and computerized order entry functionalities, low quality hospitals were generally more likely to report no concrete plans to adopt the functions, although none of the differences were statistically significant. This may have been due, in part, to the fact that the underlying rates of adoption of specific functions were high and the number of non-adopters was relatively low.

Table 3

Proportion of non-adopting hospitals with no resources or no plans to implement selected electronic functionalities in the high, intermediate, and low quality groups

The patterns for decision support functions were, however, different. We found that nearly two-thirds of all non-adopters in the low quality cohort reported no concrete plans to implement these functions, rates that were significantly higher than those reported by high quality hospitals. For example, low quality hospitals without clinical guidelines were more likely to report having no concrete plans to implement them than intermediate or high quality hospitals (67% versus 55% and 47% p = 0.02). After multivariable adjustment, the lowest quality hospitals were still significantly more likely to report no concrete plans to implement two of the key decision support tools (Appendix Table 2).

Appendix Table 2

Multivariable-adjusted Proportions and Differences in Proportions of Non-Adopting Hospitals with No Resources or No Plans to Implement Selected Electronic Functionalities in the Highest, Intermediate, and Lowest Quality Groups

Finally, when we examined hospitals’ ability to meet the Meaningful Use criteria, we found a very small percentage of hospitals across all quality categories have adopted the entire set of functions, with modest differences between them: 2.1% of high quality hospitals could meet all 9 of core measures compared to 1.1% of low quality hospitals, a difference that was not statistically significant. In sensitivity analyses, we found that the results were qualitatively similar for the alternative cutpoints (see Appendix).

When we examined individual Meaningful Use criteria, the majority were present significantly more frequently in the high quality group. Among these functions were the ability to report HQA measures to CMS (41% versus 30% and 34%, p=0.02), implement drug-drug and drug-allergy checks (25% versus 17% and 13%, p = 0.02), data exchange capabilities with other facilities (60% versus 54% and 42%, p < 0.01), and the implementation of at least one clinical decision support tool (84% versus 72% and 63%, p < 0.001) (Table 4).

Table 4

Proportion of Hospitals Meeting Selected “Meaningful Use Criteria” in the High, Intermediate, and Low Quality Grades

Discussion

We found that high quality hospitals had higher levels of adoption of nearly all EHR functions, and that the largest differences were in the presence of clinical decision support tools available at the point of care. These high-performing hospitals also had greater availability of clinical documentation tools like patient problem and medication lists. Among non-adopters, a large majority of low quality hospitals reported no concrete plans to adopt clinical decision support tools. Finally, we found that high quality hospitals were more likely to be able to meet many of the Meaningful Use criteria than low quality hospitals.

While there is a broad base of studies that have shown that EHRs can be effective in improving quality, much of the data come from a small number of pioneering facilities using home-grown EHRs.^{5, 12, 13} The failure of other studies to show a relationship between the average EHR user and quality of care benefits has led some critics to call the push for EHRs premature. Our findings suggest otherwise. We found a distinct pattern of high quality hospitals consistently using EHRs at much higher rates than low quality hospitals. These findings underscore that while EHRs alone may not transform the way care is delivered, they are likely a key, necessary component of high quality health care.

Our factor analysis has two important insights worth discussing: first, that clinical decision support tools cluster together and they do so in conjunction with CPOE, which is clinically intuitive and driven partly by the requirement that CPOE must be present for clinical decision support to be optimally effective; and second, among the highest quality hospitals, functionalities tied to viewing of clinical results more often appear together with clinical documentation functions —a pattern that was not evident in other hospitals. Whether this clustering of functions are directly related to better quality performance, or just a marker for more advanced EHR systems, is unclear and needs further investigation.

Our findings also point to the challenges ahead. Among institutions that had not yet implemented the individual EHR functions, more than half of the poor quality hospitals reported having no concrete plans for implementing CPOE for medications or several of the key clinical decision support tools. If the goal of federal policymakers is to drive improvements in care, especially among the poor performers, getting these hospitals to engage in the quality improvement process and seriously consider EHR adoption and use will be critically important. Our findings also suggest that many of the functions emphasized by the new Meaningful Use rules are already being used by high quality institutions, providing further validation to the Meaningful Use efforts as a potential way to improve quality. However, we found that only a very small percentage of all hospitals have been able to adopt all functions. Whether the millions of dollars in incentives from HITECH will be enough to achieve widespread adoption is unclear – but ensuring that all hospitals, particularly the low quality ones, focus on implementing robust decision support is critically important. Our finding that high quality hospitals are more likely to be able to meet many of the meaningful use criteria has financial implications: if HITECH does not spur poor quality hospitals to adopt EHR systems, they may fall further behind, widening the quality gulf between the best and worst hospitals.

Others have also investigated the relationship between EHR functions and quality, though none have looked for specific differences in adoption patterns between high quality hospitals and low quality hospitals. Using similar (albeit older) data, DesRoches et al. found that neither “basic” nor “comprehensive” adoption of EHR systems produced substantial gains in quality.⁸ However, this study examined the average scores among those with and without EHRs and did not examine whether EHR adoption patterns differed between the high and low quality hospitals. Himmelstein and colleagues used a dataset from the Healthcare Information and Management Systems Society (HIMSS) Analytics program and also found modest improvements in quality for those hospitals which had adopted more comprehensive computing systems compared with those with less comprehensive systems.⁹

There are important limitations to this study. First, although the HIT supplement to the AHA survey achieved a 69% response rate, non-responders were likely different than the responders, and although we attempted to statistically correct for potential non-response bias, these techniques are imperfect. Next, while we examined the adoption of specific functionalities, we had no information as to how these functionalities were used within responding institutions. This could obscure potentially important relationships between certain functionalities and quality, and we suspect that the gaps we observed between the best and worst hospitals would be even more sizeable had we been able to measure effective use of these functions. Furthermore, hospitals were not asked directly about Meaningful Use. However, our responses were mapped to analogous survey questions and our approach was generally conservative. Finally, the most important limitation of our study is the cross-sectional nature of our analysis, reducing our ability to claim a causal relationship between hospital quality and adoption of specific EHR functionalities. We did attempt to adjust for baseline differences between the quality cohorts, but as always, there could be differences in other relevant characteristics that were not measured.

Conclusions

In conclusion, we examined patterns of adoption of key EHR functions among the highest and lowest quality hospitals in the U.S. and found that high quality institutions had far greater use of most EHR functions, especially clinical decision support. These high performers were also more likely to meet many criteria for Meaningful Use. Although we could not establish that this relationship was causal, our findings suggest that for hospitals seeking to emulate care of high performing institutions, focusing on CPOE with clinical decision support is likely a key part of achieving high performance on standard quality measures. Widespread resistance to adoption, especially among low quality hospitals, points to the challenges ahead for federal policy makers as they seek to ensure that all Americans receive high quality hospital care, irrespective of where they are treated.

Appendix Table 3b

Factor Loadings for Intermediate Quality Hospitals

Appendix Tables 4a–e

Using categories grouped into top 20%, middle 60%, and bottom 20% of hospitals

Appendix Tables 5a–e

Using categories grouped into top 30%, middle 40%, and bottom 30% of hospitals

Acknowledgments

Funding Source: Dr Joynt was supported by NIH Training Grant T32HL007604-28, Brigham and Women’s Hospital, Division of Cardiovascular Medicine.

Footnotes

Author Disclosures: Dr Jha reports serving as a consultant for Humedica. The other authors (SME, KEJ, SJB) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

References

1. Department of Health and Human Services. Medicare and Medicaid Programs; Electronic Health Record Incentive Program. 2010. Vol RIN 0938-AP78.

2. Blumenthal D, Tavenner M. The “Meaningful Use” Regulation for Electronic Health Records. N Engl J Med. 2010 Jul 13; [PubMed]

3. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006 May 16;144(10):742–752. [PubMed]

4. Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood) 2009 Mar-Apr;28(2):w282–293. [PubMed]

5. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003 Jun 23;163(12):1409–1416. [PubMed]

6. Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange and interoperability. Health Aff (Millwood) 2005 Jan-Jun;(Suppl Web Exclusives):W5–10. W15–18. [PubMed]

7. P.L. 111-5 (111th Cong., 1st sess.).

8. DesRoches CM, Campbell EG, Vogeli C, et al. Electronic health records’ limited successes suggest more targeted uses. Health Aff (Millwood) 2010 Apr;29(4):639–646. [PubMed]

9. Himmelstein DU, Wright A, Woolhandler S. Hospital computing and the costs and quality of care: a national study. Am J Med. 2010 Jan;123(1):40–46. [PubMed]

10. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S. hospitals. N Engl J Med. 2009 Apr 16;360(16):1628–1638. [PubMed]

11. Jha AK, Orav EJ, Dobson A, Book RA, Epstein AM. Measuring efficiency: the association of hospital costs and quality of care. Health Aff (Millwood) 2009 May-Jun;28(3):897–906. [PubMed]

12. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999 Feb;106(2):144–150. [PubMed]

13. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. Jama. 1998 Oct 21;280(15):1311–1316. [PubMed]