|Home | About | Journals | Submit | Contact Us | Français|
Qualitative studies of the nursing home work environment have long suggested that such attributes as leadership and communication may be related to nursing home performance, including residents' outcomes. However, empirical studies examining these relationships have been scant.
This study is designed to: develop an instrument for measuring nursing home work environment and perceived work effectiveness; test the reliability and validity of the instrument; and identify individual and facility-level factors associated with better facility performance.
The analysis was based on survey responses provided by managers (N=308) and direct care workers (N=7,418) employed in 162 facilities throughout New York State. Exploratory factor analysis, Chronbach's alphas, analysis of variance, and regression models were used to assess instrument reliability and validity. Multivariate regression models, with fixed facility effects, were used to examine factors associated with work effectiveness.
The reliability and the validity of the survey instrument for measuring work environment and perceived work effectiveness has been demonstrated. Several individual (e.g. occupation, race) and facility characteristics (e.g. management style, workplace conditions, staffing) that are significant predictors of perceived work effectiveness were identified.
The organizational performance model used in this study recognizes the multidimensionality of the work environment in nursing homes. Our findings suggest that efforts at improving work effectiveness must also be multifaceted. Empirical findings from such a line of research may provide insights for improving the quality of the work environment and ultimately the quality of residents' care.
Poor quality nursing home care is a persistent public concern.1 Heightened attention to this issue2, coupled with increased regulations3, expanded consumer information4 and market incentives 5 have been slow in bringing about sustained, broad-based quality improvements.6
Emerging theories and research focusing on complex adaptive systems have suggested that relationships among co-workers within an organization impact performance.7 The Institute of Medicine study, “Crossing the Quality Chasm,” proposed a specific focus on communication, cooperation and interdependence among co-workers as factors affecting quality of care.8 Recently, nursing home “culture change” advocates emphasized the importance of the work environment as a key factor in improving organizational performance, including residents' outcomes.9, 10
Until recently, most evidence about nursing home working conditions focused largely on staffing and leadership.11, 12 Several studied took a broader view measuring communication, teamwork and leadership among nursing home staff – and establishing the reliability and validity of these measures.13-15 However, only a handful of empirical studies examined the impact of such work environment attributes on organizational performance and outcomes in long-term care. In community-based long-term care, research demonstrated the impact of these work environment attributes, measured among professionals and paraprofessionals, on perceived organizational performance16 and patient outcomes.17, 18 A few studies, which examined these relationships in nursing homes, focused on specific subgroups of workers. Relying on Directors of Nursing (DON) and registered nurses (RNs) as key informants, Anderson and colleagues showed that better communication between RNs and DON, greater participation of RNs in decision-making, and higher perception of good DON leadership among the RNs are associated with better resident outcomes.19 Most recently, Gittell et al. examined the association between relational coordination among nursing home aides and resident quality of life.20 Better relational coordination (communication, shared goals, knowledge, and respect) was found to be significantly associated with resident quality of life.
The purpose of this study is to add to this small but growing body of the literature by testing the relationships between multiple dimensions of nursing home work environment and perceived work effectiveness (performance). The utility of using perceived work effectiveness as a measure of organizational performance has been previously demonstrated in both acute21 and community-based settings,17, 18 but not in nursing homes. Secondarily, we also explore associations between performance and facility organization and structure.
The objectives of this study are to: 1) develop and test an instrument for measuring work environment and perceived work effectiveness in nursing homes; and 2) identify nursing home characteristics associated with better work effectiveness.
Shortell isolated several attributes of the work environment and showed that they influence organizational performance in acute hospitals.22, 23 In prior work, we successfully adapted the Shortell model for use in community-based long-term care.16, 18 This study builds on our prior work, but modifies the model for evaluating work environment and performance in nursing homes.
Our model identifies four domains of the work environment – leadership, communication/coordination, conflict management, and staff cohesion - as predictors of perceived work effectiveness (Figure 1). Facilities with more effective leadership, communication/coordination, and conflict management are expected to have greater staff cohesion and higher perceived work effectiveness. The literature identifies these domains as fundamental to fostering effective work environments in healthcare24, 25, including long-term care.17, 18
The domain of leadership evaluates the extent to which workers perceive their leaders to have: clarified the organization's norms and expectations, explained the chain of command, communicated their goals and expectations, and demonstrated responsiveness to changing work situations and needs. Communication and coordination measure the degree to which communication between staff members is uninhibited, accurate, timely and effective, and focuses on effectiveness of procedures for coordinating tasks and job responsibilities. Conflict management is evaluated by the degree to which parties to a disagreement communicate effectively to bring about the best possible solution, and the degree to which disagreements are brought into the open and resolved. Staff cohesion measures the extent to which workers perceive their work goals and values to be similar with those of their co-workers and the facility in which they work. These aspects of the work environment influence work performance measured by perceived work effectiveness with regard to both technical quality of care and the ability to meet the needs of residents and their families. A number of variables could impact the relationship between these dimensions of the work environment and work effectiveness: workplace conditions, resources, and staffing. Sample domain items are presented in Appendix 1. A copy of the survey tool is available upon request.
The study's secondary aim is to examine the association between facility characteristics and organizational performance. This aim is largely exploratory, as studies to date are scant, particularly with regard to the association between individual respondents' characteristics and facility performance. Our objective is to contribute to the understanding of how these factors influence perceived work effectiveness, so that future research could expand on the model presented in Figure 1. We examine several organizational (firmly within the management's sphere of influence) and structural (largely outside the management's domain) factors. They are listed in Table 1 with our hypotheses about their association with work effectiveness. We also explore several individual level variables – race/ethnicity, years of professional experience and facility service, occupation, and employment status – as potential predictors of work effectiveness.
This study utilizes data from a “parent” project that examines the impact of work performance on risk-adjusted resident outcomes in New York State (NYS). Each facility was asked to: 1) complete two management surveys (administrator and DON), and 2) distribute surveys to workers providing daily resident care. Surveys were conducted in 2006-2007. The parent study and the survey methods are described in detail elsewhere.26
In total, 7,418 surveys from direct care staff of 162 nursing homes were received (response rates ranging from 3% to 91%). In 154 of these facilities the administrators and the DONs also completed the management surveys.
The survey instrument consists of three components measuring different but complementary aspects of nursing home attributes.
The first component includes five dimensions of the work environment and performance, and two sets of control variables as depicted in Figure 1. Altogether 56 items, on a 5-point Likert scale, characterize these dimensions.
The second component includes a pre-tested, validated 12-item Likert scale from the LEAP Survey27 used in assessing nursing home management style and readiness for organizational change.28 Management style is characterized as autocratic, custodial, supportive or collegial, in order of increasing managerial receptivity for staff input, trust between management and workers, organizational capacity for change, and openness to new ideas, including teamwork.
The third component focuses on the organization of work and examines issues related to job design and presence of formally organized, multidisciplinary teams among staff providing direct resident care. Questions about the respondents' demographic characteristics and work experience were included.
The instrument was piloted using several approaches. The component measuring work environment and perceived work effectiveness was piloted with 71 nursing home staff providing daily resident care. The remaining components were also piloted but on a smaller sample of approximately 20 workers. All questions were reviewed by a specialist in education and English as a second language to confirm their appropriateness at lower than the 8th grade reading comprehension level. An expert panel of professionals/researchers reviewed the survey for face and content validity.
Domains of leadership, communication/coordination, conflict management and staff cohesion are measures of the work environment and predictors of perceived work effectiveness. In our prior work we adapted and validated an instrument for measuring these dimensions, in community-based long-term care programs, demonstrating reliability scores between 0.76 and 0.89.16
In this study, we further adapted this previously validated instrument to the nursing home setting, as follows. First, we modified the language of each item to remove references to interdisciplinary teams (focus of prior work), while retaining as much of the meaning of each statement as possible. Questions were adapted to be meaningful in the nursing home work environment and simplified to accord with the higher proportion of respondent with lower educational levels. Finally, three items were discarded as we judged them not to be appropriate for this care setting.
A numerical score, ranging from 1 (strongly disagree) to 5 (strongly agree), was assigned to each item within a domain. When 2/3 or more of the items in a domain were not completed, the response to the domain was considered to be missing. For each domain an average score was computed by adding the values of the non-missing items in a domain and dividing the sum by the number of non-missing items in that domain. A score of 5 represents the most positive and the score of 1 the most negative appraisal of a domain.
To identify factors associated with perceived work effectiveness, several facility and individual-level variables were included.
Facility Characteristics include the following variables, all of which have been defined in Table 1: management style, workplace conditions, presence of formal or self-managed teams, primary assignment, administrator and DON turnover, index of ethnic staff diversity and index of ethnic concordance of staff and residents (see Appendix 2 for detailed operational definitions), ownership status, and number of nursing hours per resident per day.
Individual Characteristics include years of experience in the facility and in the profession; race/ethnicity categorized as majority (white/non-Hispanic) and minority (all other); occupation categorized as paraprofessionals (CNAs) and professionals (e.g. nurses, physicians); and employment status (full-time, part-time, or per diem).
In addition to our prior work and assessment that the domains and items within are theoretically meaningful for nursing homes, we also employed exploratory factor analysis using the Statistical Analysis Software (SAS) PROC FACTOR procedure. Items within each scale were loaded onto a single factor. Factor loadings were evaluated using a rule of thumb that loadings =>0.30 are considered significant. Two items with values substantially below 0.30 were subsequently dropped and the analysis was repeated.
Pearson correlation coefficients were calculated between all domains to assess redundancy or conceptual independence, and between items to assess convergent-divergent validity.
To assess construct validity, we examined whether the data provide support for the theoretical model in which the relationships between the domains of interest have been defined (Figure 1). We estimated 3 regression models. In the first two models, the dependent variables were staff cohesion and perceived work effectiveness. The independent variables included the remaining dimensions of the work environment and the control variables. In the third model, the dependent variable was again perceived work effectiveness, but the independent variables were staff cohesion and the control variables. All three models included facility fixed effects to account for the hierarchical nature of the data, and for the possibility that facility-specific factors, not explicitly identified, may influence the dependent variables. We also calculated the incremental adjusted R2 when the dimensions of the work environment were added to models containing only the facility effects. This incremental adjusted R2 indicates the contribution of the work environment domains to the explanation of staff cohesion and perceived work effectiveness. These models were also estimated separately for professionals and paraprofessionals, but having found the models to be similar we only present the results for the overall model.
Reliability was assessed by measuring the internal consistency of items within each domain. Standardized Chronbach's alphas, used for this purpose, range between 0 and 1, with values exceeding 0.7 indicating high reliability.29 We evaluated the reliability of the survey using the full sample, and repeating the analysis separately for professionals and paraprofessionals, to assure that questions were not interpreted or understood differently by these two groups.
For this tool to be useful in measuring work effectiveness survey responses should have: 1) little variability across staff within a facility; and 2) substantial variability of responses across facilities. This assures that perceived performance is viewed consistently within each facility, yet is sufficiently sensitive to detect differences between facilities. We evaluated this property by calculating the F-statistic for each domain. The F-statistic is the ratio of the variation between facilities to the variation within facilities. If variability within facilities is greater than across facilities, the F-statistic will approach zero. We tested the hypothesis that F-statistic equals zero. Rejection of the hypothesis indicates that survey responses can be aggregated to the facility level and that they are meaningful in measuring differences across facilities.
To understand factors associated with work performance, we estimated two regression models. First, we estimated an individual-level model in which perceived performance was the dependent variable and characteristics of individual respondents were the independent variables. We fit a multivariate model with fixed facility effects. Because the survey data include disproportionately fewer for-profit facilities than in the general population of NYS nursing homes, we included sampling weights to obtain the appropriately weighted estimations. For all time-based variables, we included both linear and squared terms to account for the possibility that the strength of the association increases or declines over time. Such variables are highly collinear, thus inference was based on a joint F-test.
With facility's perceived work effectiveness score as the dependent variable and facility characteristics as independent variables, we fit an OLS regression model with robust standard errors estimated using STATA version 9.2. Sampling weights were also included in this model. As the response rates varied across facilities, we also estimated a model in which the response rate was included as a predictor. This allowed for the possibility that the response rate is correlated with perceived work effectiveness. As this variable was not significant and all other estimates were similar, we present the model excluding this variable.
Diagnostic tests for collinearity, using variance inflation factor were performed, but detected no evidence of significant effects that may inflate standard errors. Furthermore, based on the Breusch-Pagan and White tests we found no evidence of heteroscedasticity.
Characteristics of the 7,418 respondents and the 162 facilities are summarized in Table 2. The respondents are overwhelmingly female (91%), white (64%), with CNAs, RNs, and LPNs accounting for 50%, 19% and 14% respectively, thus reflecting the general composition of the nursing home workforce.30
All domains demonstrate good-to-high reliability (Table 3). Reliability is highest for perceived work effectiveness (0.87) and lowest for the control variables (0.73). Chronbach's alphas tend to be similar for professionals and paraprofessionals, although they are systematically somewhat higher for the former; all are within the good-to-high range of reliability. These indictors are consistent with previous work using the original validated instrument.16 Except for two items, factor loadings for each domain are between 0.366 and 0.827. Two items with values slightly below 0.300 are kept because they provide a theoretically meaningful fit. Patterns based on Pearson's coefficients show high correlations between items in the same domain and low correlation between items across domains, supporting discriminant and convergent validity of scale construction.
The results of the regression analyses undertaken to ascertain construct validity are presented in Table 4. The models we estimated confirm the associations posited in the conceptual model (Figure 1). As hypothesized, leadership, communication/coordination, and conflict resolution are positive and statistically significant predictors of staff cohesion (model 1) and, with exception of leadership, of perceived work effectiveness (model 2). Variables of workplace conditions and staffing/resources are also positive and significant predictors of these outcomes. In model 3, we examined the effect of staff cohesion and the mediating variables on perceived work effectiveness, and found all independent variables to be statistically significant.
For all the domains being assessed, the variability within is significantly lower than the variability across facilities. As measured by the F-statistics the mean of individual responses for each domain is a good approximation of responses provided within each nursing home as a whole (Table 5). This holds when tested separately for professionals and paraprofessionals.
Professionals give their facilities higher scores on the work environment domains than paraprofessionals. However, the latter assess perceived work effectiveness as being slightly better compared to the professionals. Professionals view working conditions and the availability of resources and staffing as being better, compared to paraprofessionals.
Several individual-level variables (Model 1) were significantly associated with perceived work effectiveness (Table 6). Respondents who spent more years working in a facility reported higher work effectiveness, compared to individuals with fewer years. The strength of this association decreases with time, as indicated by the square term of the variable. Respondents with longer professional experience assess work effectiveness of their facilities as being lower (F-test).
In the initial analysis occupation and race were statistically significant predictors of work effectiveness, with minorities and paraprofessionals reporting higher work effectiveness compared to majority race respondents and professionals. To further explicate these relationships we interacted race and occupation. As shown by the statistical significance of the interaction effect the minority effect is driven by the paraprofessionals. Compared with the reference group (white professionals), neither white paraprofessionals nor minority professionals show significant associations with work effectiveness. The only group with a significant effect is minority paraprofessionals (linear combinations of “race” and “race × occupation” variables at p<0.0001).
In Model 2 several factors are significantly associated with a facility's perceived work effectiveness, per hypotheses in Table 1. Nursing homes in which management style is reported to be autocratic/custodial have significantly lower work effectiveness scores compared to facilities characterized by a collegial management culture (H1). Facilities with better working conditions (H2) and higher number of total nursing hours (H9) appear to be associated with higher perceived work effectiveness. Finally, higher turnover rates of the administrators are significant predictors of lower work effectiveness (H5).
Studies of the nursing home environment have long suggested that leadership and communication are related to performance,10, 31 although few have actually tested these relationships empirically. This study presents a tool for measuring multiple domains of nursing home work environment and examines their association with organizational and structural facility attributes, and perceived work effectiveness.
Theory-based, reliable and valid measures of the long-term care work environment are much needed as workplace attributes may be key factors in improving organizational performance, including resident care.10, 32 Based on a large sample of respondents and facilities in NYS we demonstrated the reliability of the survey instrument for measuring work environment and performance, and established its face, content, and construct validity.
Our second objective, to examine individual and facility-level predictors of perceived work effectiveness, identified several potentially modifiable factors that may offer insights for improving nursing home performance.
Compared with paraprofessionals, professionals assess their work environment as being better on all four attributes. At the same time, paraprofessionals report higher perceived work effectiveness, compared to professionals. Our finding with regard to work environment is consistent with findings from other nursing home studies14 and with our prior work in community-based long-term care.16 Many reasons for dissatisfaction among the CNAs, who comprise the majority of paraprofessionals, have been identified, e.g.: managers' disregard for their input into residents' care plans; lack of trust between the CNAs and their supervisors; inadequate communication between CNAs and licensed staff; and general incongruence of goals and values between CNAs and management.32-34 The CNAs are the backbone of resident care, providing 80% to 90% of nursing care, yet nursing homes nationally experience extraordinarily high turnover rates among them.35 This very likely has a negative impact on the quality of care provided to residents. As nursing home leaders address issues of quality and staff turnover within their institutions, they may want to pay particular attention to the work environment of the CNAs. Nursing home managers should take advantage and build on the higher sense of fulfillment with work among the CNAs, because it is that perceived satisfaction with work that likely translates into better organizational performance.
Several individual and facility characteristics appear important in influencing work effectiveness. Minority paraprofessionals credit their facilities with significantly greater work effectiveness compared both to white paraprofessionals and to professionals of either race. While we have no a priori reasons to expect that staff of different socio-demographic characteristics will perceive work effectiveness differently, it is interesting to note that such differences do exist. Some support for this finding comes from other studies, which demonstrated significantly greater satisfaction with work among African-American (compared to white) nursing home workers, and among the CNAs compared to RNs and LPNs.36 Future research could examine whether the quality of care provided by minority paraprofessionals is in fact better than that provided by those of the majority race.
Other characteristics that influence the perception of work effectiveness are years in the profession and years worked at a facility. Workers with more years in the profession tend to have a worse perception of work effectiveness than those with fewer years. However, more years of work experience in a facility brings about higher perceived work effectiveness. Studies have shown that experienced CNAs, those who have learned to more effectively integrate the of their work, are more efficient and more responsive to the needs of their residents.34 CNAs with longer in facility tenure are also likely to be those with greater commitment to the residents and to caregiving as an occupation.32 Management's attention to and continuous investment in these experienced workers may be an important strategy for improving organizational performance.
Four modifiable facility characteristics are significantly associated with work effectiveness. Facilities characterized as autocratic/custodial, i.e. in which management exhibits little receptivity for input from workers or openness to new ideas, are associated with worse performance. Furthermore, good management is dependent on the stability of the leadership. The administrator and the DON are the key people in the chain of organizational and practice. Castle showed the association between higher turnover among the administrators and worse resident outcomes.37 Our study extends this finding by suggesting that it is the administrator's and not the DON's turnover that matters with regard to work effectiveness. In addition, better workplace conditions and more nursing hours are significant predictors of performance. Of the six items comprising the measure of workplace conditions, three deal with pay, benefits, and promotion opportunities. Improvements with regard to these factors, as well as higher nursing staffing, are costly and are not likely to be voluntarily implemented by nursing homes, particularly the for-profits. Early results from the recently implemented nursing home payment incentive programs in Florida and California have shown that additional payments to nursing homes did increase staff wages and resulted in some increases in staffing, but had no substantial impact on improving resident outcomes.38, 39
Our model recognizes the multidimensionality of nursing home work environments. The results suggest that efforts at improving work effectiveness must also be multifaceted, i.e. focusing on multiple work environment domains and organizational and structural attributes of the facilities.
A few caveats should be noted. First, our analysis is limited to nursing homes in NYS. Variables affected by state-specific industry regulations or environmental factors could make insights from this study not generalizable. However, neither quality of care40 nor workforce issues faced by the study facilities are substantially different from other areas of the US.30, 41 Second, assessments of the work environment and performance are based on single point in time estimates. Over longer periods of time, point estimates are not likely to accurately reflect organizational performance as individual and facility-level characteristics change.
Although perceived work effectiveness may be an important indirect measure of organizational performance, ultimately what counts is the ability to bring about good resident outcomes. The next step is to examine if facilities with better organizational performance produce better risk-adjusted outcomes for their residents. Being able to identify better performing facilities will allow for subsequent, focused assessment of their practice patterns in order to understand how good work environment brings about better quality of care.
We gratefully acknowledge funding from the National Institute on Aging, Grant R01 AG23077. Thanks also go to the participating nursing homes and their staff and to the New York Association of Homes and Services for the Aging (NYAHSA) and the New York State Health Facilities Association (NYSHFA).
[5 point Likert Scales: 1=strongly disagree…5=strongly agree]
The degree to which supervisors: are perceived as helping to develop staff capabilities to solve problems; set and communicate clear goals and expectations, facilitate their implementation, and; are responsive to changing needs and expectations.
Supervisor ignores my input when developing care plans.
Supervisor is often out of touch with our views and concerns
Supervisor provides strong guidance and advice.
The degree to which: workers feel free to speak without fear of repercussion; they believe in the accuracy and consistency of information they receive from others; resident information is relayed promptly and accurately; work activities are coordinated, and face to face interactions are perceived as effective.
There is good communication between workers across shifts.
I have received incorrect information from others in this nursing home more than once.
Co-workers are available to assist each other with patient care.
The degree to which: parties to a disagreement actively communicate and if necessary seek other expertise to resolve disputes and bring about best possible solution; disagreements are ignored; disagreements are brought to the attention of supervisors for resolution.
When co-workers disagree, they generally involve their leader in resolving their issues.
Supervisors/leaders tend to ignore disagreements between co-workers.
When co-workers disagree, they will ignore the problem pretending it will ‘go away’.
The degree to which workers: share common goals and values; identify with their nursing home; feel part of a team.
The goals and values of this nursing home are different from my own.
I take pride in being associated with my co-workers.
Our written care plans and work schedules are very effective.
The perceived effectiveness of the facility with regard to: technical quality of resident care, and the ability to meet resident and family care needs and outcomes.
We do a good job of meeting the needs of our residents' families.
We almost always meet our resident treatment goals.
Overall, my co-workers and I function very well together.
We calculated ethnic diversity as an index of decreasing majority race (DMR), defined as:
where Pw is the proportion of white and Pn of non-white co-workers. The DMR index ranges from 0.0 to 0.5. A facility composed of a single group has a DMR=0, while a facility with equal proportions has a DMR=0.5.
We calculated ethnic concordance of staff and residents as an ethnicity overlap index (EOI) defined as:
where is the proportion of residents belonging to ethnic group i and is the proportion of staff belonging to ethnic group i. When the proportion of staff and residents belonging to the same ethnic group i is the same then EOI=1. When the two groups are completely discordant, the EOI=0.5. EOI increases from 0.5 to 1 as the degree of concordance increases.