|Home | About | Journals | Submit | Contact Us | Français|
Computerized order entry algorithms can improve the quality of care; but are rarely used in nursing homes (NHs). We conducted a pilot study to: (1) develop order entry algorithms for 5 common NH problems, and (2) test their acceptance, use, and preliminary impact on quality indicators and resource utilization.
Pre-post, quasi-experimental study.
Setting: 2 Veterans Affairs NHs.
Randomly selected residents (n=265) with one or more target conditions, and 42 nursing home providers.
Expert panels developed computerized order entry algorithms based on clinical practice guidelines. Each was displayed on a single screen and included an array of diagnostic and treatment options, and means to communicate with the interdisciplinary team. Medical records were abstracted for the 6 months preceding and following deployment.
Despite positive provider attitudes toward the computerized order entry algorithms, their use was infrequent and varied by condition: Falls (73%), Fever (9%), Pneumonia (8%), UTI (7%), and Osteoporosis (3%). In subjects with falls, we observed trends for improvements in quality measures for the 6/9 measures, including measuring orthostatic blood pressure (17.5% to 30%, p=0.29), reducing neuroleptics (53.8% to 75%, p=0.27), reducing sedative-hypnotics (16.7% to 50.0%, p=0.50), prescription of calcium (22.5% to 32.5%, p=0.45), vitamin D (20.0 to 35.0%, p=0.21), and external hip protectors (25.0 to 47.5%, p=0.06). Little improvement was observed in the other conditions. There was no change in resource utilization.
Computerized order entry algorithms were used infrequently, except for falls. Further study may determine whether their use leads to improved care.
The Institute of Medicine recommends systematic employment of evidence-based clinical practice guidelines to improve the safety and quality of care.1 Evidence-based clinical practice guidelines for common problems in nursing home residents are increasingly available,2-8 but implementing them remains challenging.9-12 Barriers to implementation include limited provider knowledge of the guidelines, competing patient priorities or expectations, unclear or conflicting guideline recommendations, and lack of system-level infrastructure to support routine guideline use.13-19 In addition to these barriers, nursing homes face additional challenges including high staff turnover, limited on-site physician presence, and high regulatory requirements which conflict with or distract from guideline implementation.10, 11, 20
Computerized order entry algorithms address some of these barriers, and have improved both quality indicators and patient outcomes in acute care settings.21-23 Computerized order entry algorithms can remind clinicians of recommended steps in care, use default orders to influence treatment choice, and facilitate communication with other interdisciplinary team members through automatic consults or notifications. Nursing homes have been slow to adopt computerized medical records and order entry algorithms24-27, but several small studies suggest that they are feasible and acceptable to nursing home staff.28-32 The Veterans Affairs (VA) nursing homes (Community Living Centers) have used computerized order entry for over 10 years, and several large community nursing home chains are investing in this technology. However, the feasibility, acceptability, and impact of computerized order entry algorithms have not been widely evaluated in nursing homes.
We conducted a pilot study to: (1) develop evidence-based computerized order entry algorithms for 5 common nursing home problems, and (2) test their acceptance, use, and preliminary impact on quality indicators and resource utilization in 2 VA nursing homes.
This pilot study used a pre-post, quasi-experimental design. The Duke University, and both site VA Institutional Review Boards approved all study procedures.
Two VA nursing homes, each with over 100 beds, participated. One was academically affiliated and the other was not. Both used a combination of physicians, nurse practitioners, and/or physician assistants (collectively referred to as “providers”) to provide long-term, rehabilitation, and palliative care. While staffing varied due to turnover, both homes attempted to maintain 6 providers working in the home at a given time. In addition, 22 medical students, internal medicine/family medicine interns, and geriatric medicine fellows (collectively referred to as “trainees”) rotated through the academic nursing home during the study period.
We sought both acute and chronic clinical problems that: (1) were prevalent in nursing homes; (2) were supported by evidence-based clinical practice guidelines; (3) involved care that was primarily driven by a single provider (for example, diagnosing and treating osteoporosis is primarily accomplished by the medical provider), or required interdisciplinary coordination (for example, falls prevention requires collaboration and input from multiple disciplines). Falls,8, 33, 34 fever evaluation,3, 5 pneumonia,4, 6, 7, 35, 36 urinary tract infection,2, 3, 9 and osteoporosis37, 38 were selected.
A multidisciplinary group of experts was recruited to review the treatment evidence for these problems and determine the guidelines that were most effective, feasible, and cost-effective for the nursing home setting. These were local opinion leaders identified by the study team and local VA administrators, and included geriatricians, a nursing home medical director, an infectious disease specialist, a geriatric nurse specialist, a geriatric nurse practitioner, a registered nurse and a nurse aid who worked in one of the study homes. Attention was given to facilitating interdisciplinary communication through the computerized order entry algorithms; for example, the falls algorithm generated an automatic email alert to the facility pharmacist to request medication review. The expert panel used simple consensus to develop recommendations on the computerized order entry algorithms’ content. Based on these recommendations, 5 order algorithms were programmed into the VA computer system. The computerized order entry algorithms were used by members of the panel with test patient records, and refined in several cycles (see Appendix for final algorithms).
To facilitate implementation, 16 providers in both study facilities attended a study kick off session. During these sessions, we reviewed the goals of the project, how the computerized order entry algorithms were developed, and how they might be used to improve the quality of care. We also informed them that, to facilitate their use of these tools, we would post laminated cards on each computer station, and place reminder fliers in their mailboxes each month for the first 5 months. When new providers were hired (n=4 during the study period), study personnel met with them individually to demonstrate the use of the computerized order entry algorithms. Medical students and interns were not individually trained to use the computerized order entry algorithms, but were shown how to use them on an ad hoc basis by the medical director or geriatric medicine fellow during their rotation.
VA nursing home residents with one or more of the target conditions during the pre and post-implementation periods were identified using queries of the VA computer system followed by chart review as follows:
Residents were excluded if they did not remain in the home for at least 48 hours after the onset of an acute condition (e.g., if they were admitted to the hospital), or had advanced-directives that excluded treatment of the condition. If a subject had been included in the pre-study period for osteoporosis, they were not included in the post-study period for osteoporosis. Residents who received guideline-recommended treatment for osteoporosis in the pre-implementation period would also meet these same quality indicator criteria in the post-implementation period, and therefore limit our power to detect a true difference in the proportion of residents receiving guideline-recommended care for osteoporosis between the time periods. A convenience sample of 20 residents with each target condition was sought for each study period based on expected prevalence of the target problems; a resident could contribute to 2 conditions during a study period (e.g., had both a fever and osteoporosis). If a resident had more than 1 of the same event (e.g., 2 falls), only the first was included. When more than 20 residents with the condition were identified, a random number generator was used to select residents; otherwise, all residents with the condition were included.
Indicators were selected based on the expert panel recommendations, practice guidelines, or Assessing the Care of Vulnerable Elderly indicators (Table 3).39, 40 Resource utilization variables included frequency counts of laboratory and radiology tests ordered within 72 hours of onset for acute conditions, or within the 6 month study period for chronic conditions.
Computerized order entry algorithm use was measured by tracking automatic alerts sent to the first author whenever one was accessed. Quality indicators were abstracted from the medical record by a trained nurse or physician for the 6-month periods before and immediately after computerized order entry algorithms deployment. Using a common set of training charts, abstractors were calibrated until they achieved simple agreement on 10 key quality indicators that exceeded 90%.
Providers were surveyed at 1 and 12 months to assess their perceptions of the computerized order entry algorithms using 5-point Likert scales regarding whether they: (1) were evidence-based; (2) were easy to access and use; (3) saved time; and (4) improved the quality of patient care. Providers could write additional comments about their experience with the computerized order entry algorithms.
We present descriptive statistics on computerized order entry algorithms use, patient and problem characteristics, quality indicators, and resource utilization variables. Differences in quality indicators between the study periods were tested using Chi square or the appropriate non-parametric test. Because of the exploratory nature of the analysis, no correction for multiple testing was made.
There were potentially 200 residents per time period (target of 20 charts per condition × 5 conditions × 2 study homes). Because some residents were used for more than 1 condition, and because some conditions had fewer than 20 identified events in the time period, the actual sample size is slightly smaller. Overall, review of the records from 265 patients provided information about 276 episodes of care for the target problems. Because differing patient and disease characteristics could account for some of the observed differences in treatment of the target conditions in the 2 study periods, we collected information about these potential confounders. Patients’ characteristics were similar during the pre- (N=148) and post- (N=117) periods (table 1); and between the 2 study sites (data not shown). Patients were also similar across the two study periods in terms of the severity of the target conditions (table 2). However, 17% of pre-intervention patients, but no post-intervention patients were selected based on a diagnosis of osteoporosis on their problem list, because most of the long-term care residents with this diagnosis were already included in the pre-implementation group, and were therefore not eligible for inclusion in the post-implementation period. Therefore, the post-intervention patients were more likely to have a remote history of fracture, and less-likely to carry a diagnosis of osteoporosis.
Over the study period, 20 full-time providers and 16 trainees had access to the computerized order entry algorithms. The computerized order entry algorithms were accessed 119 times during the study period by 20 providers. All provider types, including nurse practitioners, physician assistants, trainees (medical interns and geriatric medicine fellows), and attending physicians, used the algorithms. Despite a smaller number of beds, algorithm use was greater at the academic nursing home than the non-academic site (74% vs. 26%). The falls computerized order entry algorithm was the most frequently accessed at both sites (73%), followed by fever (9%), pneumonia (8%), and UTI (7%). The osteoporosis computerized order entry algorithms was rarely accessed (3%), despite linking it to the falls algorithm 3 months after implementation. Both nursing homes experienced substantial, unanticipated provider turnover during the study period (44% in the academic-affiliated home, >100% in the non-academic home). All new staff in the academic home began using the computerized order entry algorithms immediately and no change in utilization was observed, whereas turnover in the non-academic affiliated home led to substantial declines in computerized order entry algorithm use.
Table 3 shows the observed trends in quality indicators for each target condition. Falls quality indicators improved by 10% or more in 6 of 9 quality indicators, whereas little improvement was seen in the other computerized order entry algorithms’ indicators. Several declines were observed in osteoporosis quality indicators, likely related to the significantly higher proportion of subjects with osteoporosis documented on their problem list in the pre-implementation period (table 2). Resource utilization for laboratory and radiology tests was not significantly different between the study time periods.
The computerized order entry algorithms were well-received by nursing home providers during training and at the conclusion of the study period. Because only 8 providers had > 3 months of algorithm use at the conclusion of the study and were eligible to complete the survey, we present only their qualitative responses. All 8 providers agreed or strongly agreed that the computerized order entry algorithms were evidence-based, easy to use, improved patient care and saved time. Written comments included that the computerized order entry algorithms helped remind them of all potential guideline-recommended steps, made it faster to complete the orders, facilitated communication with the interdisciplinary team, and were useful in training new staff or learners. Providers valued “being in control” of which orders were selected, which mitigated their distrust of “cookbook medicine.”
Several barriers to use were noted. Providers noted that for chronic conditions such as osteoporosis, there was no acute event or “prompt” to remind them to access the computerized order entry algorithm. Acute conditions, such as fever, frequently occurred after hours and orders were given over the phone to the nursing staff, rather than entered directly into the computer by the providers. Providers in the academic affiliated VA reported that the computerized order entry algorithms were difficult to access; although the algorithms themselves were contained on a single screen, space limitations made it necessary to have those screens accessed from a sub-folder, rather than from the main order page. Providers in the non-academically affiliated VA reported that the high provider turnover and under-staffing during the study period resulted in more verbal orders and more emergency room transfers for acute illnesses or falls, limiting the opportunities for computerized order entry algorithm use by nursing home providers.
Computerized order entry systems have the potential to improve the quality of care in nursing homes, but few previous studies have evaluated their feasibility or impact. We developed evidence-based computerized order entry algorithms for 5 common nursing home problems, and tested their acceptance, use, and preliminary impact on quality indicators in 2 VA nursing homes.
Although providers were uniformly enthusiastic about the computerized order entry algorithms, they used them infrequently. The only computerized order entry algorithm used frequently was for falls, which may have occurred for several reasons. First, falls occur more frequently than the other conditions. Second, higher staff interest in fall prevention due to regulatory and legal implications may have driven use. Finally, post-fall evaluation generally occurs during regular working hours when the providers are in the facility and can easily access the computerized order entry algorithms. In contrast, orders for acute conditions such as fever or pneumonia are often given over the telephone if they occur when a provider is not present. Osteoporosis, the only target condition without an acute event to trigger its use, was rarely used despite large numbers of patients with a history of fracture in the homes. The statistically significant decline in several of the osteoporosis quality indicators therefore likely reflects the lower proportion of patients with recognized osteoporosis in the post-implementation group. It is not clear whether providers with more experience in using computerized order entry systems would be more or less likely to use the algorithms.
When computerized order entry algorithms were used frequently, as in the case of falls, we saw a trend toward clinically meaningful improvements in quality indicators. While larger, randomized controlled studies are necessary to determine whether the change in quality indicators will translate into improved processes and outcomes for residents, this result is important because prior studies of fall-prevention quality improvement programs have not been widely successful.34, 41-44 The lack of success of previous quality improvement efforts may, in part relate to the need for coordination among multiple disciplines for multi-factorial fall risk reduction. Our computerized order entry algorithms provided a means for various groups to communicate efficiently, for example through automatic alerts sent to the pharmacist when the provider accessed the fall algorithm or through optional consults to physical and occupational therapy and nursing. Given the Joint Commission, State regulatory agency, and CMS's focus on fall prevention, we believe that the fall prevention computerized order entry algorithm merits further study as a means of post-fall assessment, and risk reduction for high risk residents.
The minimal impact on indicators for the acute and chronic conditions suggests that additional interventions might be necessary to prompt clinician use. For example, osteoporosis alerts could automatically send providers information about patients with a prior fracture history or who take chronic corticosteroids. Because providers are frequently not present in the home when an acute illness occurs, training nurses to use the computerized order entry algorithms may increase their use and facilitate team communication. However, nurses were not allowed to place orders in the computer in one of the study homes, and scope of practice limitations in other facilities would require that providers still be contacted for verbal approval. Algorithm-based standing orders for fever or falls evaluation that are automatically initiated by nursing staff is another option, although this eliminates the customization to the specific resident's presentation and goals that is valued by nursing home providers.
Provider turnover is often cited as a barrier to clinical practice guideline use in nursing homes, and was indeed observed in our study. Interestingly, the impact of provider turnover differed in the 2 study facilities. In the academic center, new providers were trained to use the computerized order entry algorithms by the medical director (who was part of the study team) during orientation, and they welcomed the computerized order entry algorithms as a valuable tool to help them “learn the ropes”. They became more frequent users than the previous providers who were accustomed to the old system. Thus, staff turnover may actually represent an opportunity to introduce new technology such as computerized order entry algorithms, particularly if the nursing home medical director and other leaders are committed and supportive. In the second facility, provider turnover exceeded 100% (including 3 medical directors) during the study period, and the facility was frequently understaffed. In this challenging environment, computerized order entry algorithm use was minimal despite repeated reminders and visits. If computerized order entry algorithms are employed by a facility, we suggest incorporating training in their use to the routine employee orientation, and actively engaging the facility leadership in promoting their use.
Our study adds to the emerging literature on the use of information technology in improving nursing home care. A recent randomized study of computer decision support to reduce adverse drug events in nursing home residents found no significant benefit.45 The authors hypothesize that limited scope of the medications covered, a high proportion of inappropriate provider alerts, and targeting of only the medical team members may have explained the lack of effect. Our study suggests that it is feasible to use computer order entry algorithms to enhance communication about geriatric syndromes with other team members (e.g., the pharmacist, therapist, and nursing staff in a patient with falls). While allowing providers to access order entry algorithms voluntarily avoids the problem of burdensome inappropriate reminders, it also may result in low use and therefore limited impact. Combining reminders to providers (e.g., reminder to screen/treat for osteoporosis triggered by an admission for hip fracture) with the appropriate computerized order entry page may allow the advantages of both strategies to be realized.
There are several limitations to this pilot study. First, it is small and the observed trends to improvement in falls quality indicators may be due to chance alone. Our pre-post study design does not allow us to make causal inferences. However, we sought to examine the feasibility and acceptance of these computerized order entry algorithms. Second, our study was conducted at VA nursing homes, which limits our generalizabitity. Notably, VA nursing homes have greater on-site availability of providers and an electronic order medical record system which may not be available in community nursing homes.
In summary, in this pilot test we successfully developed and implemented computerized order entry algorithms for common geriatric problems in 2 VA nursing homes. Despite high staff turnover, trends toward improvements in quality indicators were measured for the most frequently used computerized order entry algorithms (falls). This technology offers promise for improving clinical practice guideline use in nursing homes, but further modifications to adapt to the community nursing home system, and to prompt use for chronic conditions are needed. Formal evaluation of computerized order entry algorithms in nursing homes requires a randomized trial.
Role of the Sponsor: This study was supported by a Paul B. Beeson Award, NIA AG024787. Dr. Weinberger is a HSR&D Senior Career Scientist, Department of Veterans Affairs. The sponsor had no role in study design, data collection, interpretation, or presentation.
|Variable||Academic Nursing Home||Non-Academic Nursing Home|
|Full set of vitals except orthostatics (%)||57.1||71.4||35.0||65.0|
|Clinical Evaluation Completed within 24 hrs (%)||71.4||61.9||80.0||90.0|
|GI symptoms documented (%)||28.6||23.8||50.0||30.0|
|Urinary sx documented (%)||9.5||9.5||20.0||10.0|
|If urinary symptoms are absent AND delirium absent, then no urinalysis ordered (%)||19.1||33.3||45.0||15.0|
|No antibiotics for viral syndrome (%)||100||n/a||n/a||n/a|
|Atibiotic dose adjusted for renal insufficiency (%)||33.3||14.3||n/a||66.7|
|Only treatment for symptoms or unexplained delirium (%)||56.3||73.3||47.1||78.6|
|Recommended antibiotic prescribed (%)||62.5||26.7||64.7||64.3|
|Follow up cultures done only for symptoms (%)||37.5||60.0||40.0||50.0|
|Antibiotic dose adjusted for renal insufficiency (%)||40.0||12.5||0||0|
|Full set of vitals to include pulse ox and RR (%)||90.0||88.9||63.6||40.0|
|Clinical eval within 24 hours (%)||90.0||100.0||100.0||90.0|
|Orthostatics documented (%)||0||0||0||0|
|Recommended Antibiotic (%)||80.0||77.8||72.7||90.0|
|Antibiotic dose adjusted for renal insufficiency||66.7||100||100||100|
|Full vitals, other than orthostatics||90.0||95.0||80.0||45.0*|
|Orthostatic BP recorded in 24 hours (%)||35.0||45.0||0||15.0 (p=.07)|
|PT eval within 6 months preceeding, or 3 months after fall (%)||60.0||40.0||50.0||55.0|
|Psychotropic medication use reduced, discontinued, or justified||75.0||66.8||50.0||66.7|
|Calcium supplements (%)||45.0||60.0||0||5.0|
|Vitamin D supplements (%)||40.0||60.0||0||10.0|
|External hip protectors (%)||35.0||50.0||15.0||45.0*|
|BMD measured (%)||33.3||31.3||40.0||5.0|
|Metabolic evaluation completed|
| All of above||4.8||0||0||5.0|
|Calcium prescribed (%)||71.4||37.5*||10.0||10.0|
|Vitamin D prescribed (%)||52.4||37.5||10.0||10.0|
|External hip protectors (%)||23.8||31.3||0||0|
|Osteoporosis medication (%)||14.3||6.3||15.0||0|
|Physical Therapy Prescribed (%)||57.1||87.5*||45.0||60.0|
Conflict of Interest