|Home | About | Journals | Submit | Contact Us | Français|
In industries outside healthcare, highly skilled employees enable substantial gains in productivity after adoption of information technologies. The authors explore whether the presence of highly skilled, autonomous clinical support staff is associated with higher performance among physicians with electronic health records (EHRs). Using data from a survey of general internists, the authors assessed whether physicians with EHRs were more likely to be top performers on cost and quality if they worked with nurse practitioners or physician assistants. It was found that, among physicians with EHRs, those with highly skilled, autonomous staff were far more likely to be top performing than those without such staff (OR 7.0, 95% CI 1.7 to 34.8, p=0.02). This relationship did not hold among physicians without EHRs (OR 1.0). As we begin a national push towards greater EHR adoption, it is critical to understand why some physicians gain from EHR use and others do not.
The USA has embarked on an ambitious and expensive effort to increase adoption of health information technology (IT), with a focus on electronic health records (EHRs). The Health Information Technology for Economic and Clinical Health provisions of the 2009 American Recovery and Reinvestment Act allocate nearly $US30 billion to incentivize providers to become ‘meaningful users’ of EHRs.1 The hope is that widespread uptake of EHRs will drive improvement in both efficiency and quality of care. The evidence, however, linking the use of EHRs to lower costs and higher quality is modest at best, with several studies questioning whether there is a consistent relationship between EHR adoption and better, more efficient care.2–6
Given the push towards greater use of EHRs, understanding how this technology might be optimally used to drive improvements in care would be very useful. A potentially important source of insight may be the experience in other industries that pursued widespread adoption of IT in the 1980s and 1990s (such as banking and retail). In many of these industries, there was initially a puzzling lack of improvement in performance similar to what we are observing in healthcare. Further research into this paradox uncovered two key findings: first, there was significant heterogeneity in IT-enabled performance improvement across organizations,7 8 and second, organizations that successfully used IT to improve performance implemented complementary organizational changes that enabled them to do so. Two key organizational complements to IT were a shift towards greater use of skilled labor and an increase in decentralized decision-making.9–11 Highly skilled staff complement IT because these systems generate substantial volumes of real-time data that skilled workers can effectively turn into meaningful information. Greater decentralization of decision-making empowers these workers to immediately act on the new information in order to more rapidly improve performance.
Whether similar complementary factors affect if healthcare providers see gains from EHRs is unknown. We hypothesize that more skilled and autonomous clinical support staff play a critical role in helping physicians who have EHRs achieve high levels of performance. Such support staff may be better positioned to leverage EHR capabilities because they can assess clinical data, make decisions based on it, and execute them with minimal oversight from the physician. Physicians' use of EHRs may be optimal when they can ‘off-load’ self-contained tasks that they previously performed, such as ensuring that patients receive preventive services or adequate chronic disease management. In contrast, when physicians primarily work with lower-level staff, having an EHR may make them less productive because the physician performs many of the key activities electronically that their less-skilled staff used to perform when using paper records, a phenomenon that has been well-described.12 Even if they are able to delegate these tasks, lower-skilled staff may require more oversight and coordination, which may worsen physician performance.
Despite the importance of understanding which organizational factors facilitate performance improvement among physicians who have adopted EHRs, we lack empirical data to guide us. Therefore, using data from a survey of general internists, we sought to determine whether, among physicians that had adopted an EHR, those with highly skilled clinical support staff had higher performance on quality and efficiency than those physicians who did not use such staff. Given the possibility that having highly skilled staff may be associated with better performance irrespective of EHR adoption, we also examined this relationship among physicians who had not adopted EHRs. Gaining insight into the relationship between EHRs, organizational context, and clinical performance is relevant to both policymakers and practitioners, and the success of Health Information Technology for Economic and Clinical Health hinges on the ability to demonstrate that HIT adoption can substantively improve the efficiency and effectiveness of providers.
We used data from a survey of general internists in four states (Massachusetts, Washington, Oregon, and California). Internists were included if they fell in two performance deciles (45–55 or 90–100) on a composite quality and cost measure. A detailed description of the process used to assign this ranking is included in the next section. Within these deciles, 400 physicians were randomly selected from each of the two regions (west and east), and phone numbers were available for 235 of them. Among physicians with phone numbers, 200 internists (85% of those contacted), 100 in the top and 100 in the middle decile, agreed to participate by taking the online survey in exchange for a $US200 honorarium. The survey was developed by a team of clinicians and researchers from each of the regions. The survey instrument included a series of questions about practice demographics, structure, and our two areas of focus: IT adoption and types of clinical support staff in the practice.
Our primary dependent variable was whether the physician was in the top or middle performance decile. Assigning physicians to these deciles required leveraging multi-payer claims databases in New England and the Pacific Coast states that tracked individual internist performance on adherence to evidence-based clinical guidelines and total resource use per episode of treatment (using Episode Treatment Groups from Ingenix). Reliance on multi-payer databases helped ensure the large number of cases per physician required to meet recommendations for reliable comparison of physicians. They were also selected because the claims included standardized unit prices and thus the resource use measure primarily reflected physician practice patterns. Within each region, physicians were ranked on a composite measure that equally weighted the centile ranking on guideline adherence and low resource use per episode. (See the online Technical Appendix for additional detail.) Physicians assigned to the top performance group fell in the 90–100th centile, and those assigned to the middle performance group fell in the 45–55th centile. Our primary outcome measure compares top with middle performing physicians. We chose this commonly used approach13 14 because it elucidates how to shift the performance distribution upwards, from average performance to top-tier.
Respondents were determined to have adopted at least a basic EHR if they reported that the following five features were available in their EHR: laboratory test result viewing, radiology test result viewing, electronic visit notes, electronic medication list, and electronic problem list. These functionalities reflect the majority of required components for a basic ambulatory EHR based on consensus from a federally sanctioned expert panel15 (and all components of a basic EHR that were available on the survey).
To determine whether physicians worked with highly skilled, autonomous support staff, we drew on questions that asked physicians to report whether each of the following types of staff in their practice conducted 34 common clinical tasks (eg, obtained health history, administered medications): nurse practitioners (NPs), physician assistants (PAs), registered nurses, licensed practical nurses/licensed vocational nurses, or nursing assistants/medical assistants. Based on their length of training and scope of practice, we considered NPs and PAs to be highly skilled, autonomous staff and determined that they worked with the physician if the physician reported that NPs or PAs performed one or more of the tasks on the task list.
We identified a set of covariates that could potentially confound the relationship between physician performance and presence of NPs or PAs for physicians with and without EHRs. These included physician gender, whether the physician owned their practice, whether the physician was in solo practice, and whether the physician was in the east- or west-coast cohort.
We began by comparing differences between high and medium performers. We then examined bivariate relationships between performance decile and our variables of interest (presence of NPs/PAs, EHR adoption, and the four controls) among physicians who had adopted EHRs and then among physicians who had not adopted EHRs. We subsequently built multivariate logistic regression models to determine whether the presence of highly skilled, autonomous support staff (NPs or PAs) was related to physician performance, again looking first among physicians with EHRs and then among physicians without EHRs to validate that the relationship was specific to those with EHRs. To assess whether the relationship between presence of NPs/PAs and performance decile was statistically different for physicians with and without EHRs, we ran a third model with physicians that had adopted EHRs and those that had not adopted EHRs, which interacted EHR adoption and presence of NPs/PAs.
When we compared our two samples of physicians, we found very small, statistically insignificant differences between high (top decile) and moderate (middle decile) performers in their likelihood of working with highly skilled, autonomous staff (67% vs 59%, respectively; p=0.24) (table 1). Similarly the two groups were not distinguishable in their likelihood of having at least a basic EHR (30% and 39%, respectively; p=0.18) (table 1).
In the bivariate analysis among physicians who had adopted an EHR, we found that having highly skilled, autonomous clinical support staff was far more common among high performers than among moderate performers (OR=5.0, p=0.02; table 2). In the multivariate model that adjusted for physician gender as well as key practice characteristics (size, ownership, location), we found that among physicians with EHRs, those with highly skilled, autonomous clinical support staff had seven times greater odds of being a top performer (OR=7.0, 95% CI 1.4 to 34.8, p=0.02; table 2).
Among physicians who had not adopted EHRs, we found very different results. In the bivariate relationship, there was no association between having highly skilled staff and being a top performer (OR=1.1, 95% CI 0.5 to 2.2, p=0.83; table 2). In the multivariate model, those with highly skilled, autonomous clinical support staff were no more likely to be a top performer than those without these staff (OR=1.0, 95% CI 0.5 to 2.0, p=0.99; table 2). When we formally tested whether the relationship between having highly skilled staff and being a high performer varied by EHR adoption status, we found a significant interaction (p=0.03). We also found that physicians who own their own practice were less likely to be a top performer, which is consistent with prior literature,16 although this did not differ for physicians with and without EHRs.
We found that the presence of highly skilled, clinically autonomous staff was associated with substantially greater odds of being a high performer on quality and efficiency among physicians with an EHR. This relationship did not hold for physicians without EHRs. These findings, the first of this type in healthcare of which we are aware, are consistent with the IT effectiveness literature from other industries. While the study was small in scope, the sizeable association suggests that many of the same phenomena that have led to better use of IT in other industries may also be at play in physicians' offices. Given that most providers will eventually switch to using EHRs, the key policy question is how we ensure that providers leverage EHRs to improve performance. Here, the data on organizational ‘complements’ is particularly useful and novel.
Why might having more skilled and autonomous clinical support staff play a critical role in helping physicians who have EHRs achieve high levels of performance? Although we cannot answer this question directly with our data, we believe that more advanced clinical support staff may be better suited to using the EHR to identify opportunities to improve care. Perhaps more importantly, those able to make decisions and act autonomously can address newly identified opportunities without the need to coordinate with a physician. While there may be alternative mechanisms that underlie our findings, those we suggest are consistent with what has been found across a range of other industries that heavily invested in IT.9 17 Since our study is among the first to empirically examine these issues in healthcare, further assessments will clearly be necessary to validate these findings. If our results are confirmed, the next challenge will be to develop approaches to ‘upskill’ clinical support staff. One option is to train new NPs and PAs, and we estimate that tens of thousands of new staff would likely be needed to ensure that each ambulatory practice employed at least one. Alternatively, practices can better train currently employed lower-level support staff to perform EHR-specific tasks with minimal oversight.
Beyond understanding the role of clinical support staff in EHR effectiveness, it will be important to explore more broadly why some providers realize substantial gains from EHRs while others struggle. Such inquiries will be useful to clinical leaders, policymakers and front-line practitioners. For policymakers, our finding points to the importance of expanding NP and PA training programs as well as considering new types of training programs focused on enhancing IT-related capabilities of lower-skilled clinical support staff. For clinical leaders as well as front-line physicians, our results suggest that more highly skilled and autonomous staff may play an important role in ensuring that EHRs improve performance, and their jobs should be designed to enable them to do so.
There are important limitations to our work. First, we had a relatively small sample of physicians and therefore lacked power to find smaller differences, although this should not invalidate the associations that we did find. Second, although we tried to account for potential confounding, we had a limited set of variables and therefore could not include all potentially relevant covariates. This is a problem with nearly all observational data, and, in order to be sure that organizational complements really were causally associated with better performance, we would probably need a randomized trial. Finally, we currently lack the empirical research to help us understand differences in how more highly trained, autonomous staff interact with EHRs. As a result, the mechanisms we propose are largely speculative. They have, however, been widely discussed in the IT literature outside of healthcare, and they are critically important to assess in the healthcare setting.
In conclusion, we used the knowledge of IT effectiveness in other industries to assess whether similar organizational complements to IT apply in healthcare. We find preliminary evidence that more highly skilled, autonomous clinical support staff may serve as an important complement to EHRs in enabling high levels of physician performance. This is consistent with the notion that IT is best used by staff who can leverage newly available data to make decisions and then execute them autonomously. While larger, confirmatory research is needed, we believe our findings have important implications for how work should be redesigned after EHR adoption. Clinical leaders and policymakers may need to pay greater attention to the significance of organizational factors to ensure that adoption of health IT substantively improves health system performance.
Contributors: JA-M and AJ contributed to the planning, conduct, and reporting of the work.
Funding: Robert Wood Johnson Foundation and California HealthCare Foundation.
Competing interests: None.
Ethics approval: Massachusetts.
Provenance and peer review: Not commissioned; externally peer reviewed.