|Home | About | Journals | Submit | Contact Us | Français|
To test the utility of a two-dimensional model of organizational climate for explaining variation in diabetes care between primary care clinics.
Secondary data were obtained from 223 primary care clinics in the Department of Veterans Affairs health care system.
Organizational climate was defined using the dimensions of task and relational climate. The association between primary care organizational climate and diabetes processes and intermediate outcomes were estimated for 4,539 patients in a cross-sectional study.
All data were collected from administrative datasets. The climate data were drawn from the 2007 VA All Employee Survey, and the outcomes data were collected as part of the VA External Peer Review Program. Climate data were aggregated to the facility level of analysis and merged with patient-level data.
Relational climate was related to an increased likelihood of diabetes care process adherence, with significant but small effects for adherence to intermediate outcomes. Task climate was generally not shown to be related to adherence.
The role of relational climate in predicting the quality of chronic care was supported. Future research should examine the mediators and moderators of relational climate and further investigate task climate.
Effective chronic disease management requires the collaboration of patients and providers in the context of supportive health systems (Wagner, Austin, and Von Korff 1996). Although health services research increasingly emphasizes the impact of organizational context on health care outcomes (Yano et al. 2007), the role of contextual factors in chronic disease management has received relatively little study. Chronic diseases have a broad range of severity and many patients are managed in primary care settings (Rothman and Wagner 2003). In this paper, we focus on organizational climate as an important contextual feature of primary care settings relative to the management of chronic diseases. Specifically, we report results of an investigation of whether organizational climate as defined by collective perceptions of the work environment (James et al. 2008) is positively associated with the probability that patients receive care that adheres to process and intermediate outcome guidelines for a chronic illness. Our investigation is one of the first such studies to link organizational climate to objective clinical outcomes.
Organizational climate is based on perceptions of policies, procedures, and practices. Climate is often conceptualized as a mediator linking organizational characteristics to attitudes and behaviors (James et al. 2008). As a concept it is similar but distinct from organizational culture, which is typically considered to represent the underlying assumptions and values of the organization (Scott et al. 2003). We focus on organizational climate because it has been shown to have potential relevance for the effective management of both chronic and preventive care (Warren et al. 2007). Moreover, a meta-analysis found support for relationships between organizational climate and customer loyalty, turnover, and productivity in a variety of business settings (Harter, Schmidt, and Hayes 2002). Our investigation advances the literature by examining the relationship between specific dimensions of organizational climate and specific quality measures for chronic care covering both process and intermediate outcomes.
For this study, we applied a conceptual framework in which human perceptions and behavior are broadly organized under task and relational dimensions (Benzer and Meterko 2010). Task climate refers to a management focus on achievement and improvement. Relational climate refers to a management focus on mutual support and respect. This framework is consistent with a long tradition of research in the psychology and management literature (Halpin and Winer 1957; Alderfer 1972; Borman and Motowidlo 1993; Gittell 2002;). As discussed below, both dimensions are potentially predictive of a primary care team's performance in managing chronic conditions.
The management strategy implied by task climate potentially affects the management of chronic conditions in two ways. One is through identifying and rewarding performance goals. Goal-aligned performance is influenced by factors such as assigned goals, rewards, and performance feedback (Locke 2001). Management of chronic diseases may involve multiple process and outcome goals (Wagner et al. 2001). For example, organizations with a strong task climate are more likely to promote performance by providing performance scores (feedback) to clinical personnel and basing financial rewards on those scores.
Task climate may also affect chronic care by influencing the way in which tasks are performed. Assuming that adherence to best practices for chronic disease is a performance goal, goal-focused primary care clinics are likely to develop systems and procedures focused on chronic disease management (e.g., education programs). Thus, we hypothesize that task climate is positively associated with chronic care management.
The management strategy implied by relationship climate can contribute to effective chronic care by promoting trust and supportive collaborations among staff. Organizational research supports the general value of relationship-oriented strategies in terms of their impact on performance (Dirks 1999; Bachrach et al. 2006; De Dreu 2007;). In clinical settings specifically, effective interactions between providers and support staff have been shown to promote adherence to chronic care guidelines (Wagner 2000; Wrobel et al. 2003;). By fostering positive interactions among staff, a relationship-oriented climate may enhance communication and coordination among nurses, physicians, and other caregivers in the management of patients with chronic disease.
Many patients require education, guidance, and social support to change their health behaviors (Gensichen et al. 2009). Clinics with supportive interactions can adjust procedures based on differences in patient motivation to help patients actively participate in their own care (Hayes et al. 2008). Ensuring that care is tailored to the specific needs of patients necessitates collaboration of nurses, social workers, nutritionists, and others with both primary and specialty care physicians (Wagner 2000; Ostbye et al. 2005; Zwar et al. 2006;). As such, a relationship-oriented climate may help promote effective self-care among patients with chronic disease. Accordingly, we hypothesize that relational climate is positively associated with chronic care management.
This study was conducted in the U.S. Department of Veterans Affairs (VA). VA provides subsidized medical care to veterans, and it is organized in regional networks divided into hospitals and freestanding outpatient clinics. An advantage of VA as a research setting is that clinical facilities operate under a common set of fiscal and administrative rules and regulations. Therefore, investigations of VA clinical facilities are less likely to be biased by varying economic and environmental factors that can influence the performance of health care delivery organizations. VA managers also have considerable discretion in managing their organizations through setting individual performance goals and financial rewards, development and design of clinical teams, adoption and implementation of clinical policies and procedures, and human resource practices. Accordingly, there is opportunity for differences in management strategy among primary care clinics within VA.
This study analyzed secondary data involving 4,539 patients across 223 primary care clinics and 21 networks. We included hospital-based and freestanding outpatient clinics. Data were analyzed at the patient level with patient and clinic covariates. Employee climate ratings were aggregated to the clinic level (average 30 employees). Only direct care providers were included (e.g., physicians, nurses). Bliese (1998) demonstrated that the bias in reliability with small sample sizes is greatest for group sizes smaller than five. Therefore, groups with fewer than five respondents were excluded. Overall, 282 unique primary care clinics were linked to the patient sample. Of these clinics, 223 could be identified by a primary care workgroup with at least five survey respondents. Most of the unmatched workgroups (K = 32) were located in freestanding clinics, likely too small for five primary care respondents. Chi-square tests revealed no significant differences in diabetes processes and outcome measures between matched and unmatched clinics.
Guideline adherence was measured through VA's External Peer Review Program, an independent chart review of randomly selected patients (VA Office of Quality and Performance 2007). Sampling was not developed for the clinic level of analysis and thus smaller clinics were expected to have small sample sizes. Data were collected quarterly; this study used the diabetes data collected between October 2006 and September 2007.
Organizational climate was measured through the All Employee Survey (AES), an annual voluntary survey of all VA employees, collected in May 2007 (71 percent participation). The AES consists of three parts, respectively, focusing on individual, workgroup, and facility organizational levels. We used task and relational climate measures from the workgroup-focused section. Multiitem scales were adapted from the U.S. Office of Personnel Management employee survey (Gowing and Lancaster 1996) and subsequently refined across three prior survey administrations and validated with both employee-level job attitudes and facility-level employee and patient outcomes (Warren et al. 2007; Benzer and Meterko 2010; Meterko et al. unpublished data).
We defined primary care clinics as multidisciplinary groups of health care professionals providing acute, preventive, and chronic care, who are recognized as an organizational unit within the health care system in which they work. Primary care clinics were identified using two sources. First, patients were assigned to primary care physicians and clinics in administrative records. Second, primary care clinics were identified using the AES planning process where AES coordinators in consultation with senior managers at each VA facility designated local workgroups with shared goals and responsibilities. Workgroups were included in analyses if specifically identified as having a primary care function. Employees were instructed to respond to the AES referring to their designated workgroup.
Task and relational climate were measured with three items each on a 5-point, Likert-type scale using a workgroup referent (Appendix SA2). Task climate (ICC = 0.11) and relational climate (ICC = 0.11) had adequate intraclass correlations for clinic-level research. Using the Spearman-Brown prophecy formula, ICC of 0.072 suggests a group mean reliability of 0.70 (Bliese 2000). The two-factor model of climate was tested using two-level confirmatory factor analysis using MPLUS Version 5.2 (Muthén and Muthén 2007). The model demonstrated good fit to the data (χ2(26) = 193.07) with factor loadings over 0.70 at the individual and clinic level of analysis, and alternative fit indices (CFI = 0.98; RMSEA = 0.03; SRMRwithin = 0.03; SRMRbetween = 0.03) within typical thresholds (March, Hau, and Wen 2004; Sivo et al. 2006; Chen et al. 2008;). The latent variable correlation between the two climate constructs was above 0.80. High intercorrelations between climate dimensions are typically attributed to a general climate factor representing a global evaluation of the unit (James and James 1989). Researchers often collapse multiple dimensions into a single climate factor (Brown and Leigh 1996; Stone and Gershon 2006;), but prior research using task and relational climate demonstrated that a single factor obscures important differences between the climate constructs (Benzer and Meterko 2010). Internal consistency estimates were adequate for task (αindividual = 0.87; αclinic = 0.93) and relational climate (αindividual = 0.86; αclinic = 0.94).
As noted, we used diabetes-related data for assessing adherence to chronic illness guidelines. Diabetes is a representative example of a highly prevalent chronic illness that requires a high degree of coordination among multiple health care providers (Malpass, Andrews, and Turner 2009). Diabetes is particularly prevalent in veteran populations (Congressional Budget Office 2009). Specific care processes with empirically documented relationships to clinical outcomes have been clearly articulated and agreed upon in diabetes care. Focusing on diabetes clinical outcomes avoids a potential confounder reflecting differences in providers' perspectives on best practices and variation in treatment procedures (e.g., depression, schizophrenia). The widely accepted, empirically supported key features of diabetes care include control of health indicators, for example, blood pressure and cholesterol, and treatment for lifestyle factors, for example, weight reduction, smoking cessation, exercise, and nutritional habits (Puder and Keller 2003). Nevertheless, there are numerous potential barriers for adherence to clinical practice guidelines in diabetes (Larme and Pugh 1998; Luftey and Ketcham 2005; Ratsep, Oja, and Kalda 2007;), and organizational interventions may influence the provision of diabetes care (Piette et al. 2000; Grant and Meigs 2006;).
We measured care processes with adherence to annual HbA1c test requirements and foot examinations. We measured intermediate outcomes through adherence to clinical standards for HbA1c<9, LDL-C<120, and blood pressure <140/90. All measures used dichotomous patient-level adherence scores (1 = adherence). Laboratory test scores represent the most recent values when the record was selected for review. No HbA1c or blood pressure values in the past year and no LDL-C in the past 2 years were coded as nonadherence. Foot and HbA1c exams indicate at least one record of the procedure in the prior year. There were 87.6 patients per clinic (SD = 72.9), with over 200 patients in 21 clinics, and 12 clinics with <10 patients.
Analyses were conducted with SAS PROC GLIMMIX (SAS 9.1.3, Cary, NC) to account for the four-level nesting of patients within physicians, clinics, and networks. Analyzing the effects of clinic-level variables on patient-level data directly models the heterogeneity of patients and avoids ecological fallacies for patient-level inferences. Variables were entered into the model stepwise starting with the patient control variables, then the clinic control variables, relational climate, and finally task climate. The stepwise method provides incremental estimates of effect sizes generated using the procedure detailed in Appendix SA3 (Snijders and Bosker 1999).
In multilevel modeling, group-level random effects are determined through a weighted average of both the group-level data and an empirical Bayes estimate derived from the population mean. The influence of the group-level data increases with larger sampling; random intercept modeling therefore adjusts for the problem of low reliability for clinics with a small number of sampled patients (Snijders and Bosker 1999). Nevertheless, clinic processes could qualitatively differ in clinics with fewer patients; therefore, sensitivity analyses were conducted using only clinics with a minimum of 25 patients (80 percent of the sample).
Patient-level covariates included age (centered at 65.26, the population mean), comorbidities, with dummy variables for sex (1 = male) and marital status (1 = married). The veteran population tends to be male and elderly, and clinic processes may be tailored to this population (Congressional Budget Office 2009). We adjusted for marital status as a proxy for social support. Comorbidities were measured with the clinical classification software (HCUP 2010). Scores were calculated as the total number of comorbid conditions.
Organizational variables included clinic setting, complexity, geographic location, and turnover rate. Clinic setting identifies freestanding clinics with hospital-based clinics as the comparison. Complexity is classified by VA based on seven criteria, including patient population, clinical services, education, and research into five categories (VHA Facility Complexity Workgroup 2005). Complexity accounts for differences in the demands and resources available to clinics of similar size and is modeled using dummy variables with the lowest complexity level 3 as the comparison group. Geographic location accounts for regional differences among the northeast, central, and southwest United States using the western region as the comparison. Turnover rate was included as an indicator of staffing stability, potentially impacting teamwork independent of relational climate (Zhang et al. 2007), and also as a proxy for affective constructs such as job satisfaction.
Descriptive statistics are presented in Table 1. Correlations between adherence measures were generally low (tetrachoric correlation < = 0.20). The number of diabetes patients sampled for each clinic varied by outcome, as not all patients were eligible for all measures. Across the quality indicators, the average number of patients sampled for a clinic ranged from 17 to 21, and the median patient age ranged from 56 to 83. For the diabetes process measures, 4,485 foot examinations were conducted for 4,827 eligible patients (93 percent), and 4,700 HbA1c tests for 4,539 eligible patients (97 percent). For the diabetes intermediate outcomes, 4,215 patients of 4,539 eligible patients (87 percent) met HbA1c control guidelines, 3,959 patients of 4,539 eligible patients (82 percent) met LDL guidelines, and 3,980 patients of 4,849 eligible patients (82 percent) met blood pressure guidelines.
Table 2 presents results from statistical analyses of the incremental effect of patient control variables (model 2), clinic control variables (model 3), relational climate (model 4), and task climate (model 5) over the intercept-only model (model 1) for each of the quality indicators. The proportion of clinic-level variance to total variance was adequate for both foot exams (ICC = 0.06) and HbA1c testing (ICC = 0.05), but more variance was accounted for at the provider level. Table 3 presents odds ratios for the model with all variables as predictors. Estimated odds ratios indicated that relational climate was positively associated with the likelihood of patients receiving foot exams and HbA1c tests. Adding task climate in step 4 revealed an unexpected negative relationship with HbA1c exams and increased the estimated odds ratio for relational climate from 1.48 to 2.22. An examination of the control variables revealed that male veterans were up to four times more likely than female veterans to receive annual foot exams, and HbA1c tests were more likely to be conducted in hospitals compared with freestanding outpatient clinics, and in medium complexity facilities.
Intraclass correlations in Table 2 indicated that clinic-level variables are not likely to explain a large proportion of variance in intermediate outcomes. Blood pressure control was found to demonstrate the highest proportion of clinic-level variation (ICC = 0.03), with minimal clinic-level variation observed for HbA1c and LDL-C control. Patient control variables were significantly related to the likelihood of patient's compliance with both the HbA1c and LDL-C standards, with married veterans being more likely to be in compliance for all outcomes and limited evidence for an effect of sex, age, and comorbities. Clinic factors were not related to patient compliance with the exception of a negative relationship with blood pressure control for highly complex facilities.
Relational climate was positively related to blood pressure (odds ratio = 1.41). The positive effect of relational climate on HbA1c control (odds ratio = 1.28) became nonsignificant when task climate was included. Relational climate was not significantly related to the LDL-C intermediate outcome. In models without relational climate, task climate was not significantly related to either process or intermediate outcome measures.
Sensitivity analyses were conducted by restricting clinics to at least 25 sampled patients per clinic (80 percent of the sample). No differences were observed for this reanalysis, indicating that including clinics with a low number of eligible patients did not influence results. Post hoc analyses were also conducted on a restricted sample of veterans age 50 or older to investigate the relationship between age and the intermediate outcome measures. The odds ratio associated with age did not change in the restricted sample, indicating that the relationship between age and the intermediate outcomes measures is due to older veterans being more likely to be compliant with the standards rather than younger veterans being less likely to be in compliance. It is possible that a correlation between diabetes outcome adherence and longevity is driving this observed effect. Finally, we examined the order of the stepwise method used and verified that task climate was not related to outcomes with relational climate omitted.
Results provide partial support for a relationship between organizational climate and primary care clinic effectiveness. Relational climate was positively related to an increased likelihood of patients receiving annual foot inspections and HbA1c tests, with smaller effects observed for maintaining blood pressure at recommended levels. Observed effect sizes were not large compared with those observed for patient-level variables, but as previously noted climate is conceptualized as an indirect measure of the social interactions and other organizational dynamics that may promote chronic disease management. Relational climate was observed to be a robust predictor of high-quality diabetes care across process measures. The documented benefits of positive relational climates are broad, and now include objectively measured diabetes care in addition to employee satisfaction and well-being (Carr et al. 2003). Given that the patient-level outcomes are generally not highly correlated, we suggest that the consistent observed relationship between clinic climate and improved patient care, particularly for process measures, is noteworthy and warrants future study.
The lack of significant findings for task climate suggests that a management emphasis on assigned performance goals and associated rewards may not be very effective for improving care for chronic conditions such as diabetes. In models without relational climate, task climate was not associated with outcomes, and in models with both climate measures, task climate was only related (negatively) to blood pressure control. Prior research suggests that assigned performance goals can decrease extra-role behavior, suggesting that a task focus not balanced by a relationship focus could be counterproductive (Wright et al. 1993). Identifying possible limitations of task-oriented strategies is particularly important given the growing emphasis that health plan executives and policy makers are placing on pay-for-performance programs to improve the quality of care (Tanenbaum 2009). Indeed, much of the research to date addressing pay-for-performance in health care suggests that the implementation of these financial incentive programs is often not very effective for improving quality (e.g., Petersen et al. 2006; Rosenthal and Frank 2006; Young et al. 2007;). Our results suggest that even a broad platform of performance goals and financial rewards that is reflected in a task-oriented climate may not contribute much to quality improvement, at least not without other organizational arrangements in place such as strong relational strategies.
Several alternative explanations are also available as to the lack of an observed effect for task climate in our study. First, although local VA leaders have considerable discretion in the assignment of performance goals and distribution of rewards, it is possible that the system's overall emphasis on diabetes-related goals may obscure effects of task climate. Second, it is possible that unmeasured moderator variables affect the relationship between clinic-level task climate and individual-level improvement efforts. As noted earlier, diabetes is a complex disease that requires the coordination of multiple providers and involves multiple short-term as well as long-term goals. The number of diabetes-related goals could dilute the observed relationship with climate as individual providers may be more likely to focus on the goals most relevant to individual patients' immediate needs and postpone work on goals that are not seen as urgent.
A third explanation is that the measure of task climate may not have been specific enough to identify the variation in level of clinic focus on diabetes-relevant goals compared with other primary care goals. Researchers might focus on more narrow measures of task climate with a focus on specific goals such as diabetes care. The organizational literature suggests that a focus on climate relative to specific goals is likely to improve predictive validity (Schneider, White, and Paul 1998), but the use of general climate scales is more practical for administrative use due to the large number of important health care goals.
The organizational climate conceptual framework offers several benefits. First, climate has been actively studied in the organizational literature for over 40 years. Methodological advances such as hierarchical modeling have provided the means to rigorously examine the relationship between clinic climate and performance (Morgeson and Hofmann 1999; Chen, Bliese, and Mathieu 2005; Mathieu and Taylor 2007;). The current study demonstrates the potential relevance of climate research to guideline-based care. Effective management of health care clinics requires an awareness of climate as a contextual factor that contributes to effective care and serves as a reliable indicator of potential problems. Future research should investigate causal relationships of social behaviors in the clinic context with effective patient care, but in the interim, relational climate may serve as a convenient proxy for health care managers to evaluate the quality of teamwork related to chronic disease management.
Relational climate is related to other group-level constructs like cohesion, and it would likely be related to interpersonal behaviors such as communication, collaboration, and information sharing. Practically, climate variables are important to health care managers because employee surveys tend to measure climate constructs (i.e., perceptions of managerial practices). This study demonstrates that administrative surveys can be used to reliably measure climate-related variables, and items that measure the social environment can be linked to guideline-based care.
In addition to organizational interventions targeting clinic procedures (Piette et al. 2000; Grant and Meigs 2006;), clinical leaders may consider implementing organizational interventions focused on interpersonal interactions in the workplace. VA has developed an intervention to improve the quality of interpersonal interactions in the workplace (Osatuke et al. 2009). Other health care organizations might consider developing similar interventions or employing external consultants to improve the quality of the workplace relational climate.
As an observational study, our results are sensitive to several threats to internal validity. Our high response rate indicates that nonresponse bias was mitigated, but nonresponse does introduce the potential for systematic differences between respondents and nonrespondents, particularly for those clinics with small numbers of employees. In an observational, cross-sectional study, we also cannot account for endogeneity, as achievement of goals could possibly influence climate perceptions. Although we discuss the hypothetical causal role of climate, we note that the current data do not support causal inferences. We also acknowledge that facilities may systematically differ by patient or organizational factors. For example, chronic disease management requires coordination between clinics for complicated conditions; coordination between units is distinct from within-unit relational climate and will have unique barriers and facilitators. This limitation notwithstanding, the results are still compelling. The diabetes outcomes have generally low correlations, and a spurious effect is unlikely across multiple outcomes. These findings provide an impetus for a more rigorous investigation of both clinic setting and relational climate on care of diabetes and other chronic conditions. Future research should examine the role of organizational climate over time, including the mediating role of coordination processes to identify best practices in primary care clinic management.
Another limitation concerns the generalizability of the results; the study was conducted entirely in one public health care delivery system and for one chronic disease in primary care clinics. We suggest that relational climate may be important in the care of any disease that requires coordination and collaboration. Research is needed to test the results in other settings, particularly within the private sector. Such research would be very timely because health care reform proposals have emphasized a medical home model (AAFP 2009), with collaboration among providers being touted as critical components of successful implementation. The adoption of a medical home model would increase the importance of teamwork in patient care and thereby the importance of relational climate as a determinant of patient outcomes (Rosenthal 2008).
Finally, observed effect sizes were small. The use of dichotomous indicators does reduce the explainable variance, but if future researchers hypothesize a causal effect of relational climate, they should consider mediating and moderating variables. Potential mechanisms include mediating effects whereby improving climate increases coordinated care within clinics through a normative environment that encourages cooperation and collaboration. In this role, relational climate would likely have the strongest effect on patient outcomes when those outcomes require coordinated efforts from multiple health care staff, such as the literature suggests is needed for effective chronic disease management. It is also likely that climate only serves as an indicator of social interactions rather than a cause, and thus interventions should focus on improving coordination and communication behaviors within clinics.
It is also possible that climate has a moderating role with chronic disease management, whereby interventions focusing on social processes such as coordination and communication will be most effective when climate provides a strong normative foundation for prosocial behaviors. A metaanalysis of chronic care interventions indicated that organizational interventions have a positive effect on outcomes and processes of care but a wide confidence interval suggests the presence of moderators (Tsai et al. 2005). Relational climate is a good candidate for a moderator, particularly for interventions such as team practice and care coordination. From this perspective, it is not surprising that the main effect of relational climate would be small, as heterogeneity in delivery system design may obscure effects. Determining the causal role of climate on patient outcomes requires intervention research. We suggest that researchers should include relational climate when planning comparisons between intervention and control sites, or include relational climate in the intervention design (e.g., Osatuke et al. 2009) and investigate the causal role of climate through studies of organizational change.
In conclusion, the chronic care model suggests that patient behavior can be influenced by provider and system-level factors. Relational climate may be an important factor in determining the effect of organizational design on patient and provider behavior. Leaders considering implementing a medical home model should consider the relational climate of their organization. We propose that leaders can promote de jure coordination through mechanisms such as integrated leadership or shared workspace, but relational climate is an important factor in de facto coordination of care.
Joint Acknowledgment/Disclosure Statement: This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development (IIR 05-221). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States Government. Financial support was provided in part from a VA Office of Academic Affiliations HSR&D postdoctoral fellowship and a Robert Wood Johnson Foundation Investigator Award in Health Policy Research. An earlier version of this manuscript was presented at the 2010 Academy of Management Annual Meeting. The authors acknowledge the intellectual and editorial contributions of James Burgess and Mark Bauer. No conflicts of interest influenced this manuscript.
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Appendix SA2: Organizational Climate Scales.
Appendix SA3: Specification of Multilevel Models.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.