|Home | About | Journals | Submit | Contact Us | Français|
To examine quality improvement (QI) implementation in nursing homes, its association with organizational culture, and its effects on pressure ulcer care.
Primary data were collected from staff at 35 nursing homes maintained by the Department of Veterans Affairs (VA) on measures related to QI implementation and organizational culture. These data were combined with information obtained from abstractions of medical records and analyses of an existing database.
A cross-sectional analysis of the association among the different measures was performed.
Completed surveys containing information on QI implementation, organizational culture, employee satisfaction, and perceived adoption of guidelines were obtained from 1,065 nursing home staff. Adherence to best practices related to pressure ulcer prevention was abstracted from medical records. Risk-adjusted rates of pressure ulcer development were calculated from an administrative database.
Nursing homes differed significantly (p<.001) in their extent of QI implementation with scores on this 1 to 5 scale ranging from 2.98 to 4.08. Quality improvement implementation was greater in those nursing homes with an organizational culture that emphasizes innovation and teamwork. Employees of nursing homes with a greater degree of QI implementation were more satisfied with their jobs (a 1-point increase in QI score was associated with a 0.83 increase on the 5-point satisfaction scale, p<.001) and were more likely to report adoption of pressure ulcer clinical guidelines (a 1-point increase in QI score was associated with a 28 percent increase in number of staff reporting adoption, p<.001). No significant association was found, though, between QI implementation and either adherence to guideline recommendations as abstracted from records or the rate of pressure ulcer development.
Quality improvement implementation is most likely to be successful in those VA nursing homes with an underlying culture that promotes innovation. While QI implementation may result in staff who are more satisfied with their jobs and who believe they are providing better care, associations with improved care are uncertain.
Improving the quality of nursing home care is a national priority (Institute of Medicine 2001). As with other sectors of the health care industry, nursing homes are increasingly applying industrial quality control principles to the task of improving care (Zinn, Brannon, and Weech 1997). These management principles, known as continuous quality improvement, total quality management, or simply quality improvement (QI), emphasize developing a structured, organization-wide approach to understanding and improving underlying work processes (Blumenthal 1993). Quality improvement implementation in nursing homes, though, has not been extensively studied. Limited past research suggests that more than three-quarters of nursing homes practice some quality improvement activities and that the adoption of these practices is influenced by both institutional and market factors (Zinn, Weech, and Brannon 1998). However, these results were based on surveys of nursing home administrators and may not capture the true extent of QI implementation within the organization. Additionally, the effects of QI implementation on the adoption of best practices in the nursing home and on improving resident outcomes are uncertain (Institute of Medicine 2001). We therefore set out to examine QI implementation, its association with organizational culture, and its relationships with care within nursing homes maintained by the Department of Veterans Affairs (VA).
The VA operates a federally financed health care system for eligible veterans. The system comprises more than 150 medical centers, many of which have associated nursing homes. Although part of a larger medical center, VA nursing homes have a separate administrative structure consisting of a medical director and a director of nursing. Nursing home units within the VA are also often physically separated from the rest of the medical center, occupying separate buildings or campuses. Beginning in the early 1990s, VA headquarters began to encourage its hospital directors to adopt QI as “an essential management method and an organizational imperative” (Kizer 1999; Young, Charns, and Shortell 2001). Yet previous research suggests that the implementation of QI is a gradual process and considerable differences have been found among VA medical centers, as well as private sector hospitals, regarding the extent to which QI practices are being followed (Parker et al. 1999; Shortell et al. 1995). Additionally, questions have been raised as to whether QI can be successfully implemented in nursing homes (Kane 1998; Institute of Medicine 2001). We specifically test the hypothesis:
H1: Veterans Affairs nursing homes differ in their implementation of QI practices.
Determinants of quality improvement implementation have been actively investigated. Zinn and colleagues (1998) have hypothesized that nursing homes adopt QI as a management tool in response to environmental pressures as conceptualized by resource dependence and institutional theory. They found that perceived competition, the Medicare share of hospital discharges in the market, and the Medicare census within the nursing home were associated with QI adoption. These factors, though, are likely to be less important in the VA.
Studies from nonnursing home settings have demonstrated that organizational culture, those values, beliefs, and norms of an organization that shape its behavior, is an important determinant of QI implementation (Parker et al. 1999; Shortell et al. 1995). Shortell et al. reported that the implementation of QI practices among hospitals is associated with a group/developmental culture, that is, a culture where innovation, risk-taking, and teamwork are highly valued. This finding makes sense intuitively since in health care delivery organizations, QI represents a substantial departure from traditional quality assurance methods. For a health care organization to implement QI successfully, employees must be willing to take chances and learn new ways of doing their work. Moreover, because QI in health care settings requires communication among employees from different clinical disciplines, a culture that emphasizes teamwork also seems essential. The generalizability of this earlier research to nursing homes, where much care is provided by aides with little formal education and clinical training, is uncertain. We therefore test the hypothesis:
H2: A greater degree of QI implementation will be seen in those nursing homes with the strongest group/developmental culture.
In concept, QI implementation empowers employees to be actively involved in all aspects of care. It encourages staff to develop innovative practices that may improve care. In hospital settings, QI implementation has been associated with managements' perception of improved human resource development, which included ability to recruit and retain staff, physician commitment to the hospital, and nursing staff satisfaction (Shortell et al. 1995). Similar effects are likely in the nursing home. We hypothesize that:
H3: Quality improvement implementation is positively associated with employee satisfaction.
Quality improvement implementation is closely related to clinical practice guidelines (James 1993; Mittman, Tonesk, and Jacobson 1992; Burns et al. 1992). Guidelines are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific circumstances” (Field and Lohr 1990). Quality improvement encourages an analytic, evidence-based approach to medical care. Important steps in the QI process include identifying appropriate guidelines to solve problems and removing barriers to their implementation. Guidelines provide the initial description of best practices while QI supplies a set of tools to iteratively implement and customize them to a particular setting (James 1993). Nursing homes that are further along in implementing QI practices may be more likely to be using guidelines and to have adopted the best practices that they recommend. We now test the hypothesis:
H4: Quality improvement implementation is positively associated with guideline adoption and the performance of best practices.
Considerable uncertainty exists regarding the association between QI and patient outcomes (Shortell, Bennett, and Byck 1998). There have been few randomized clinical trials of QI and the results have generally been disappointing. Moreover, questions are increasingly being raised regarding the feasibility and generalizability of such studies (Samsa and Matchar 2000; Goldberg 2000). Consequently, most hospital-based studies either have evaluated QI as a method for generating interventions or have examined observationally the association between extent of QI implementation and outcomes.
Similar approaches, although limited in number, have been applied to nursing homes. Several clinical trials of QI have not been encouraging (Schnelle et al. 1998; McKenna, Moyers, and Feurberg 1998). Additionally, Zinn et al. (1997) found no association between QI adoption as determined from a survey of nursing home administrators, and two outcome measures: percent of residents with pressure ulcers and percent with contractures. However, these outcome measures were limited in that prevalence (as opposed to incidence) of these conditions may be more influenced by admission patterns than by care actually provided in the nursing home, and the measures were not risk-adjusted. We address this issue in testing the hypothesis:
H5: Quality improvement implementation in nursing homes is associated with better risk-adjusted patient outcomes.
Study hypotheses are presented schematically in Figure 1. This figure is not intended to contemplate all possible relationships among study variables, but rather the relationships that are the focus of the present study, with QI implementation being the central variable. In testing these hypotheses, we focused on a single clinical outcome as a general indicator of quality of care, pressure ulcer development. Pressure ulcers are common in nursing homes, are associated with considerable morbidity and mortality, and are often preventable through the provision of appropriate interventions that require the coordinated care efforts of multiple providers (Berlowitz and Wilking 1993). Best practices related to pressure ulcer care have been widely disseminated through clinical practice guidelines published by the Agency for Healthcare Research and Quality (AHRQ, formerly Agency for Health Care Policy and Research) (Panel for the Prediction and Prevention of Pressure Ulcers in Adults 1992). There is a general consensus that pressure ulcers are related to quality of care, and those nursing homes with high rates of pressure ulcer development may be more likely to have problems with other quality measures and with deficiency citations during state inspections (Rudman et al. 1993; Mukamel 1997). Moreover, information on pressure ulcer development is available from VA databases, and risk-adjustment models for this outcome have been developed and extensively tested (Berlowitz et al. 1996; Berlowitz, Young, Brandeis et al. 2001). Thus, pressure ulcers may be an ideal measure in assessing nursing home quality.
We initially selected 40 VA nursing homes for possible inclusion in the study. Nursing homes were selected so as to meet the following three characteristics. First, they were to represent diverse geographic areas. Second, they were to vary in size, with approximately half having fewer than 100 beds. Third, nursing homes were to differ in their quality of care as measured by the rate of pressure ulcer development calculated from an administrative database. Two nursing homes were dropped from this initial list because we could not identify a local liaison to assist with data collection, two were dropped due to delays in obtaining institutional approval, and one was dropped from the analyses due to its exceptionally small size, with resulting limited information on performance. Thus, our final sample consisted of 35 nursing homes.
Data used in this study were obtained from three sources. Information on organizational culture, quality improvement implementation, guideline adoption, and satisfaction was obtained from a survey of nursing home employees. Performance of best practices related to pressure ulcer prevention was obtained through medical chart reviews. Risk-adjusted rates of pressure ulcer development were calculated from an administrative database.
We surveyed a sample of employees from each of the 35 nursing homes. Employees at each nursing home were selected from a list of clinical staff provided by the nurse-administrator at each nursing home. From this list, we identified our survey sample consisting of all nurse managers, attending physicians, and nurse practitioners for each nursing home, as well as a random sample of remaining staff including registered nurses, licensed practical nurses, nursing aides, dieticians, and physical therapists. We determined the total number of staff to be surveyed at each site according to the formula 100 X [N/(N+100)] where N is the total number of clinical staff at the nursing home. The use of the formula resulted in our sampling higher proportions of staff from smaller nursing homes and lower proportions from larger homes. For example, a nursing home with a staff of 100 would have 50 individuals selected to receive the survey whereas a nursing home with a staff of 300 would have 75 individuals selected to receive the survey.
A liaison at each nursing home distributed the surveys to the study sample beginning in July 1998. Enclosed with each survey was a letter from the liaison requesting the employee's participation, a letter from the investigators ensuring confidentiality of responses, and a stamped return-envelope. A reminder postcard was mailed out two weeks later and nonrespondents received a second survey approximately four to six weeks after the initial distribution. This second survey was mailed directly to staff so that the local liaison would have no knowledge of which staff responded to the survey. A third survey was mailed to nonresponding nurse managers as well as to staff at eight nursing homes with low initial response rates.
The survey contained approximately 100 questions related to quality improvement, organizational culture, and supervisory practices. Copies of all survey instruments may be obtained from the corresponding author. The extent of quality improvement implementation in a nursing home was measured using 42 items that operationalize the Malcolm Baldridge National Quality Award Criteria. The VA worked with a consulting firm to develop the instrument for assessing QI implementation (Parker et al. 1999), which is similar to an instrument used by Shortell et al. (1995) in their previously noted study on QI implementation in hospitals. Items are divided into five subscales that address how nursing homes improve the quality of their service through leadership, information analysis, strategic quality planning, human resource management, and management of process quality. Each item allows respondents to indicate their level of agreement with the statement using a 5-point Likert scale. A sample question from each of the five subscales is presented in Table 1. In previous research on VA medical centers, the five subscales have demonstrated good internal consistency with Cronbach alphas ranging from 0.89 to 0.92. For each respondent, we calculated the mean of the 42 items. To measure the extent of quality improvement implementation for a nursing home, we aggregated the scores of respondents by each nursing home in the sample (i.e., we calculated the mean of the scores for all responding employees). Higher scores on the resulting 1 to 5 scale indicate a greater extent of QI implementation.
Organizational culture was assessed using a 20-item instrument developed by Zammuto and Krakower (1991). The instrument was originally developed in non-health care settings but has since been applied to hospital care (Shortell et al. 1995; Parker et al. 1999). The instrument consists of five sections that relate to either an organization's focus or people. For each section respondents are asked to distribute 100 points among four descriptions of culture that best characterize their own organization. The four types of culture are: a group culture that emphasizes teamwork and shared decision making, a developmental culture that emphasizes innovation, a rational culture that emphasizes planning and productivity, and a hierarchical culture that emphasizes rules and regulations. Each nursing home potentially contains elements of all four cultural types. By distributing a total of 500 points among the four culture types, respondents indicate which type of description best explains the culture of their organization. The more points allocated to a particular cultural type, the stronger its perceived emphasis in the nursing home. Following Shortell et al. (1995), the points allocated to the group and developmental culture types were combined to create a culture score for the instrument. We then computed a single score for individual nursing homes by aggregating the combined group/developmental score across respondents. We also recalibrated the scores so that the maximum number of points was 100 rather than 500. The closer the total number of points are to 100, the greater the emphasis in the nursing home on a group/developmental culture.
Each survey also asked employees about their familiarity and adoption of five AHRQ clinical practice guidelines relevant to nursing home care. For these analyses, we make use of the single response to the question of whether the pressure ulcer prevention guideline was adopted. Adoption was defined as indicating that “guideline recommendations were integrated into your unit's daily practice.” Staff-reported adoption of the guideline was then described as the percentage of respondents at each nursing home reporting its adoption. Employee satisfaction was captured using a 5-point scale in response to the question, “Overall, how satisfied are you in your job?”
We identified best practices related to pressure ulcer prevention from the previously noted AHRQ clinical practice guideline. To determine nursing home use of the best practices, specially trained nurses abstracted nursing home medical records. We examined medical records from residents of study nursing homes who were admitted during 1998, did not have a pressure ulcer on admission, and remained institutionalized for at least one week. We reviewed 20 to 30 medical records at each site with two-thirds of the selected records being for residents at high risk of ulcer development and one-third at low-risk. The instrument used for record abstractions was originally developed by RAND as part of its Quality and Utilization Review Criteria Project (Laouri et al. 1995). We modified the instrument for use in nursing homes. The instrument uses information from medical, nursing, and ancillary staff notes during the first six weeks of care. The instrument is used to identify residents with specific characteristics such as immobility or incontinence as well as when, during the first six weeks of care, the characteristic was first identified. We then determined whether and when specific recommended care processes from the guideline were actually carried out. Up to 15 guideline recommendations could be indicated in any resident. Adherence to guideline recommendations was described as the percent of indicated processes documented in the medical record for the full set of 15 recommendations as well as among the 6 recommendations rated most important by study investigators.
Our methods for calculating the outcome measure of pressure ulcer development has been described previously (Berlowitz et al. 1996; Berlowitz, Young, Brandeis et al. 2001). Briefly, we determined rates of pressure ulcer development using the Patient Assessment File (PAF), a VA administrative database originally developed for case-mix-based reimbursements in nursing homes according to Resource Utilization Groups version 2 (RUGs II) (Schneider et al. 1988). Information in the PAF is collected on all VA long-term care residents at the time of admission and semi-annually on April 1 and October 1. Thus, long-staying residents will have six months between assessments. By following a resident over time in the database, it is possible to identify changes in health status, such as the development of a pressure ulcer. We defined pressure ulcer development as present when a resident without an ulcer on an “index” assessment had a stage 2 or larger ulcer on a subsequent assessment. Residents with stage 1 ulcers were considered pressure ulcer-free. Risk-adjustment was performed using a previously derived and validated model that considers 11 resident characteristics. Data used in this study were from three, six-month periods that overlapped our chart abstractions, beginning in October 1997 and ending in April 1999.
Descriptive statistics were calculated for each study variable including QI implementation, organizational culture, staff-reported guideline adoption, employee satisfaction, and adherence to guideline recommendations. Analysis of variance was used to test for differences among nursing homes in their extent of QI implementation (H1). Linear regression models were used to test associations among the 35 nursing homes between their scores on these variables. Specifically, we examined each nursing home's organizational culture score in relation to its QI implementation score (H2) and each nursing home's QI implementation score in relation to each of employee satisfaction (H3), staff reported guideline adoption (H4), and adherence with guideline recommendations (H4). Analyses relating QI implementation to pressure ulcer development (H5) were performed at the patient-level using a random effects model. Each resident was assigned the QI implementation score of his or her nursing home as the independent variable. The dependent variable (O-E) was the observed rate of pressure ulcer development (1 if a pressure ulcer developed or 0 if not) minus the expected rate (as predicted by the logistic model) for each individual patient.
We further examined these associations by including a limited number of nursing home characteristics as independent variables in the regression models. These included region of the country (East, South, Midwest, and West), nursing home size defined by number of beds, urban versus rural location based on presence in a Metropolitan Standard Statistical Area, and teaching status according to membership on the Council of Teaching Hospitals. As all rural facilities were nonteaching, these last two variables were combined as urban-teaching, urban-nonteaching, and rural. Due to the relatively small number of facilities, each characteristic was used as an independent variable in separate regression models testing each of H2, H3, and H4.
In examining the results of these analyses, two limitations of our methods should be considered. First, our data were all cross-sectional, capturing the process of QI implementation at one point in time. Thus, caution is required in assuming causal direction among study variables, and the associations reported in the paper may have occurred through different mechanisms than the ones we assumed to exist in advancing our hypotheses. Second, the employees that responded to our survey completed several instruments contained in the same questionnaire that potentially addressed overlapping concepts. Accordingly, our measures may have been vulnerable to response set bias. Efforts to replicate our results with measures obtained from independent data sources will be valuable contributions to the literature.
A total of 1,781 clinical staff of VA nursing homes were surveyed. Completed surveys were received from 1,065 staff, for an overall response rate of 60 percent. Response rates by position in the nursing home are described in Table 2. The response rate was highest for nurse managers, with 72 of 88 (82 percent) completing the survey, and lowest for physicians, with only 48 of 88 (54 percent) completing the survey. The overall response rate by nursing home ranged from 39 to 85 percent with 31 of the 35 nursing homes having a response rate of 50 percent or greater. Respondents were typically experienced health care workers; 96 percent of the nurse managers, 75 percent of the nurses, and 71 percent of the nursing aides reported more than 10 years experience in the field.
There were considerable differences among nursing homes regarding their extent of QI implementation, which supports hypothesis 1. The mean (±SD) score on the 42-item QI implementation survey was 3.55±0.72. However, mean scores at individual nursing homes ranged from a low of 2.98 to a high of 4.08 (p<.001). Quality improvement implementation was significantly greater (p=.003) among nursing homes located in the West compared to the South and East (mean scores 3.93, 3.61, and 3.50, respectively). No differences in QI implementation was evident based on facility size, teaching status, or urban versus rural location.
Nursing homes also differed considerably in their organizational culture. Their mean (±SD) culture score, which as noted combines the group and developmental scores from the instrument, was 44.9±23.1 with a range from 29.3 to 64.9. In support of hypothesis 2, those nursing homes with a stronger group/developmental culture had a greater degree of QI implementation. In the linear regression model, a 10-point increase in the organizational culture score was associated with a 0.17 increase in the QI implementation score (model R2=0.31; p<.001) (Table 3).
Employees were generally satisfied regarding their jobs. The mean (±SD) satisfaction score on the 5-point scale was 3.68±1.03 and the range among nursing homes was from 3.19 to 4.18. Nursing homes with a greater degree of QI implementation also had on average a higher level of satisfaction among their employees, supporting hypothesis 3. A 1-point increase in the QI implementation score was associated with a 0.83 increase in the satisfaction score (model R2=0.57, p<.001) (Table 3).
Study results offered mixed support for hypothesis 4. We found a statistically significant association between the extent of QI implementation and staff-reported adoption of the pressure ulcer prevention clinical practice guideline. A 1-point increase in the QI implementation score was associated with a 28 percent increase in staff reported adoption (model R2=0.31, p<.001) (Table 3). However, we also used chart review to assess actual performance of clinical practices recommended in the guideline. No association was found between a nursing home's QI implementation score and the adherence score based on either the 15 guideline recommendations we examined or the 6 most important ones (p>0.2 for both analyses).
Study results offer limited support for hypothesis 5. Rates of pressure ulcer development among the 35 nursing homes ranged from 0 to 9.9 percent. In the random effects model, the association between QI implementation and the risk-adjusted rate of pressure ulcer development was in the expected direction (i.e., negative), although the p-value was 0.19 (Table 3). The random effects equation estimated was: O−E=6.02−1.71X(QI Implementation). This indicates that individuals at nursing homes with lower than average QI implementation scores would be more likely than expected to have developed pressure ulcers, while residents of nursing homes with higher than average QI implementation scores would be less likely than expected to develop ulcers. This observed excess risk drops by 0.17 for every 0.10 increase in the QI implementation score. Adherence to guideline recommendations as determined by chart review was not associated with staff-reported adherence; neither measure was associated with pressure ulcer development.
Associations demonstrated above were generally unchanged when regression models were repeated including the nursing home characteristics as independent variables. Estimates were of similar magnitude and most of the nursing home characteristics were not significantly associated with the dependent variables.
The Institute of Medicine (2001) in its recently published report Improving the Quality of Long Term Care, identified QI as one method by which nursing homes could improve care. However, the report also acknowledged the lack of evidence regarding the effectiveness of QI in nursing homes and the difficulty in implementing QI in a setting characterized by limited trained staff and organizational capacity. We performed a comprehensive evaluation of QI implementation in nursing homes maintained by the Department of Veterans Affairs to examine its association with organizational culture and its effects on care.
It is important to recognize that VA nursing homes differ from community nursing homes in two important ways. First, unlike many community nursing homes, VA nursing homes are part of a larger integrated health system that has made a significant investment in QI. Thus, VA nursing homes have access to information and staff resources that may not be available to independent nursing homes. However, a growing number of community nursing homes are becoming part of large chains and health systems and thus, in the future, may be more similar to VA nursing homes in terms of information systems and specialized staff. Second, nursing aides within the VA have less turnover than their counterparts in community nursing homes and, thus, may be more experienced. For example, 71 percent of the aides reported more than 10 years experience in health care with most of the experience obtained within the VA. These nursing aides may be more accepting of QI practices or better able to adopt these practices. Nevertheless, issues faced currently by VA nursing homes in implementing QI, such as how to focus on consumer needs, implement guidelines, and improve processes of care, are likely to be similar to the issues faced by other nursing homes. The VA may just be further along in implementing recommendations from the Institute of Medicine report in terms of organizational capacity and staff development.
In testing H1, we found significant differences among VA nursing homes in their implementation of QI practices. Mean scores on the 5-point implementation scale at individual nursing homes ranged from less than 3 to greater than 4. This finding is consistent with those of Zinn et al. (1998) in noting that there are differences among nursing homes in their level of interest in and commitment to QI practices. Our results indicate that some VA nursing homes have been more successful in responding to the system-wide initiative promoting QI. Success is not dependent on size, teaching status, or urban location. We cannot be certain why nursing homes located in the West had greater QI implementation.
Moreover, our results suggest that QI implementation is related to an underlying organizational culture. Consistent with our hypothesis (H2), those nursing homes with a stronger group/developmental culture appeared to be further along in implementing QI practices. Our results are similar to those reported by Shortell et al. (1995) in their study of hospitals. In our study, the mean group/developmental score of nursing homes was 45 (of 100) versus 46 in their study; a 10-point increase in the organizational culture score was associated with a 0.17 point increase in QI implementation in our study versus 0.18 in their study. Our results indicate that QI cannot just be implemented in any nursing home. Rather, the nursing home must be suitably predisposed to QI by having a culture that rewards innovation and teamwork. Efforts to implement QI in nursing homes without such a culture are less likely to be successful.
Our results highlight some of the benefits of QI implementation. Employees at nursing homes that had adopted more QI practices were significantly more satisfied with their jobs. Moreover, the magnitude of this effect appeared considerable. QI implementation may enhance satisfaction by empowering employees to be more active in daily care decisions. These results are consistent with those from hospitals reported by Shortell et al. (1995). They noted an association between QI implementation and human resource development as described by a scale that captures factors such as ability to recruit and retain clinical staff, nursing staff satisfaction, and employee turnover. Because of this prior work, we were especially interested in the association between QI implementation and satisfaction. However, it is possible that this effect is mediated by other factors that jointly influence QI implementation and employee satisfaction.
We also found that staff at nursing homes with a greater degree of QI implementation believed that they were doing a better job, as represented by their statement of having adopted best practices contained in the pressure ulcer guideline. Zinn et al. (1997) also noted that administrators of nursing homes that had adopted QI perceived a positive impact of QI on quality of care and resident satisfaction. Yet we were unable to show that pressure ulcer preventive practices, as documented in the medical record, were actually better at nursing homes with more QI practices. We cannot be certain whether this truly indicates no differences in practices or simply the difficulty in detecting key practices, such as turning patients every two hours, in the medical record. Given how little is known about how guidelines are implemented in nursing homes (Berlowitz, Young, Hickey et al. 2001), further evaluation of the association between QI implementation and the process of care is needed.
Ultimately, it is the effect of QI on patient outcomes that is of critical importance. Recent articles have highlighted the limited literature in this area as well as the uncertain effects of QI on care (Shortell, Bennett, and Byck 1998; Shortell et al. 2000). There are few clinical trials of QI implementation specific to nursing homes (Institute of Medicine 2001). One study directed at improving the management of urinary incontinence found initial improvements that were not sustained (Schnelle et al. 1998). Other randomized clinical trials of QI and pressure ulcer care have either reported negative results (McKenna, Moyers, and Feurberg 1998) or are still underway (Gifford 1999). Concerns have been raised, though, as to whether QI can be assessed using randomized clinical trials. Thus, some studies, including ours, have considered an alternative approach that uses observational data from many nursing homes to measure the extent of QI implementation and determine its association with outcomes. Notably, Zinn et al. (1997) found no association between QI adoption and the facility percentage of residents with urinary catheters, physical restraints, pressure ulcers, or contractures. In comparison to Zinn et al., the strengths of our study included the ability to study the incidence of pressure ulcer development rather than prevalence, the ability to consider patient-level factors in modeling, and the availability of an extensively validated risk-adjustment model for our outcome of interest.
Despite these strengths, we were unable to demonstrate a significant association between QI implementation and the risk-adjusted rate of pressure ulcer development. We did observe, however, a relationship that was in the expected direction whereby residents of nursing homes with a higher degree of QI implementation had a lower pressure ulcer rate. Furthermore, the effect size was clinically meaningful with a 1-point increase in the facility QI implementation score (approximately the difference between the nursing homes with the highest and lowest scores) being associated with an observed pressure ulcer rate 1.7 percent lower than the expected rate. If we had had a larger sample of nursing homes, and thus more statistical power, this effect might have been significant. Interestingly, neither staff-reported guideline adoption nor pressure ulcer preventive practices, as documented in the medical record, were associated with the rate of pressure ulcer development. The absence of such associations could also be due to inaccuracies in the data on pressure ulcer development. Although studies evaluating the PAF have been limited, good agreement among nurse-reviewers has been reported (Rudman, Bross, and Mattson 1994). These results highlight the difficulties in trying to link process and outcome measures.
It is increasingly important to understand not only which nursing homes are doing well, but also how some nursing homes are achieving better outcomes (Brannon 1992). We have now examined one method nursing homes may employ to improve the quality of their care. We have found that there are differences among nursing homes in their implementation of QI practices, and that QI appears to be associated with employee satisfaction and the perception of providing better care. However, our results were inconclusive in terms of demonstrating an effect of QI on quality of care. Thus, our results emphasize the need for continued study before QI is widely promoted as a means for improving nursing home quality. Our results do seem to support the importance of developing an organizational culture and capacity for implementing QI. Quality improvement, as well as other interventions to improve care, is unlikely to be successfully implemented in nursing homes that are not suitably predisposed to making the necessary changes in how care is delivered.
The study team gratefully acknowledges the contribution of Boris Kader, Ph.D., in performing the analyses, Cheryl Burton, R.N., in performing chart abstractions, and Bruce Ferrell, M.D., in assisting in the creation of the medical records abstraction instrument.
This work was supported by the Department of Veterans Affairs, Health Services Research and Development Service, (VA)grant no. CPG 97-012.