PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Care. Author manuscript; available in PMC 2013 August 21.
Published in final edited form as:
PMCID: PMC3748716
NIHMSID: NIHMS457866

Associations Between Organizational Characteristics and Quality Improvement Activities of Clinics Participating in a Quality Improvement Collaborative

Sarang Deo, PhD,* Keith McInnes, ScD, MSc, Charles J. Corbett, PhD, Bruce E. Landon, MD, MBA, MSc,§ Martin F. Shapiro, MD, PhD, Ira B. Wilson, MD, and Paul D. Cleary, PhD**

Abstract

Background

Few studies have rigorously evaluated the associations between organizational characteristics and intervention activities of health care organizations participating in quality improvement collaboratives (QICs).

Objective

To examine the relationship between clinic characteristics and intervention activities by primary care clinics that provide HIV care and that participated in a QIC.

Design

Cross-sectional study of Ryan White CARE Act (now called Ryan White HIV/AIDS Treatment Modernization Act) funded clinics that participated in a QIC over 16 months in 2000 and 2001. The QIC was originally planned to be a more typical 12 months long, but was extended to increase the likelihood of success. Data were collected using surveys of clinicians and administrators in participating clinics and monthly reports of clinic improvement activities.

Measures

Number of interventions attempted, percent of interventions repeated, percent of interventions evaluated, and organizational characteristics.

Results

Clinics varied significantly in their intervention choices. Organizations with a more open culture and a greater emphasis on quality improvement attempted more interventions (P < 0.01, P < 0.05) and interventions that were more comprehensive (P < 0.01, P < 0.10). Presence of multidisciplinary teams and measurement of progress toward quantifiable goals also were associated with comprehensiveness of interventions (P < 0.01, P < 0.05).

Conclusion

Clinic characteristics predicted intervention activities during a QIC. Further research is needed on how these organizational characteristics affect quality of care through their influence on intervention activities.

Keywords: organizational structure, quality improvement collaborative, chronic disease

Problems with the quality of health care for chronic conditions14 have stimulated interest in quality improvement (QI) methods.56 One widely used QI method is the quality improvement collaborative (QIC). A fundamental premise of this approach is that most quality problems are due to short-comings in the organization and management of health care delivery systems rather than individuals.711 Some QICs focused on chronic conditions have used the Chronic Care Model (CCM) to guide their activities.1224 The CCM describes 6 fundamental areas that need to be focused on to improve chronic care processes and outcomes: the delivery system, patient self-management, decision support, information systems, community linkages, and health system support.

Despite the popularity of QICs, the evidence regarding their impact on care quality is mixed.1420 Some organizations participating in a QIC achieve dramatic improvement while others achieve modest or no improvement.25,26 The factors affecting decisions that organizations make regarding improvement activities during QICs need to be understood better.27 Recent evaluations of QICs have found significant heterogeneity in the QI activities of organizations,15,18,20,21 but have not investigated how these activities are associated with organizational characteristics. This association is strong in other QI contexts.2833

We used data from a national QIC focused on HIV care at Ryan White Clinics15 to assess the extent to which clinic characteristics, such as QI focus and openness of organizational culture, are associated with the number and comprehensiveness of interventions adopted by participating clinics.

METHODS

Data

The data for this study were collected by the Evaluation of Quality Improvement for HIV Care project,15 a national evaluation of a QIC involving 62 HIV clinics that received funding under Title III of the Ryan White CARE Act. The collaborative was conducted over 16 months between 2000 and 2001 by the Institute of Healthcare Improvement.

Data on clinic characteristics were collected using screening questionnaires that were administered when clinics were recruited for the QIC and responses to surveys of administrators and randomly selected clinicians administered (response rate of 89%) at the beginning of the QIC in 2000. Data on interventions attempted during the QIC were collected from monthly reports submitted by the participating clinics.

Of the 54 clinics that provided data to the Evaluation of Quality Improvement for HIV Care project, we analyzed data from 41 clinics, which provided intervention data and responses to the clinician and administrator surveys. We had survey data from 119 clinicians (an average of 2.8 per clinic) and 41 administrators.

Measures

Number of Attempted Interventions

As part of the QIC, each clinic submitted monthly reports describing the QI interventions attempted. Data corresponding to each intervention reported in each monthly report were coded on a separate data entry form. Each intervention was categorized under one or more of the 6 CCM components (eg, implementing a patient reminder system under Delivery System Redesign, implementing computerized reminders for providers under Decision Support). We used these data to calculate the number of interventions attempted as a measure of the intensity of implementation or overall activity level during the QIC.18,20,21

Comprehensiveness of Attempted Interventions

Wagner et al have argued that chronic care improvement requires attention to all 6 components of the CCM.23 Other studies have assessed comprehensiveness by counting the number of CCM components addressed in the interventions21 or by using the number and percentage of interventions attempted for each CCM component.20 For this study, we developed indices that measure breadth (whether the clinics attempted interventions in a narrow or a broad subset of the CCM components) and depth (the extent to which individual interventions, on average, addressed multiple CCM components simultaneously), and comprehensiveness (breadth and depth combined) after controlling for the total number of attempted interventions (Appendix A, http://links.lww.com/MLR/A40).

Organizational Culture

Previous articles in both the organizational and health services literatures suggest that leadership commitment to QI21,22,31,32 and a supportive organizational and interpersonal climate that encourages employees to engage in experimentation3436 are important organizational characteristics associated with successful QI efforts. To measure these aspects of clinics, we used responses to 9 questions in the clinician surveys about the leadership and staff attitudes towards implementing QI initiatives that were developed for this study. We averaged the responses of clinicians within each clinic; exploratory factor analyses on the aggregate responses identified 2 factors that we labeled “Organization QI focus” and “Openness of organization culture” (Appendix B, http://links.lww.com/MLR/A40).

Other Organizational Characteristics

We used responses from administrator and clinician surveys that were developed for this study to construct dichotomous variables for 2 organizational characteristics considered important in QI: regular use of multidisciplinary teams (including nonmedical staff)24,33,37,38 and regular measurement of progress toward quantifiable goals,25 respectively. Screening questionnaires were used to collect data on clinic type (community-based organization, community health center, university medical center, public health clinic or part of a larger multispecialty hospital), whether the clinic was a specialty clinic focusing on HIV or a general practice, number of HIV patients, and region (South, West, Northeast, and Midwest).

Analyses

We estimated a count regression model to identify significant predictors of the number of interventions attempted by each clinic. Preliminary descriptive analyses indicated overdispersion of the dependent variable, so we used a negative binomial model.39 We estimated linear regression models to identify the predictors of the breadth, depth, and comprehensiveness of the interventions, defined earlier. The independent variables used in these models were QI focus, openness in organizational culture, measurement of quantifiable goals, and presence of multidisciplinary teams. In these models we controlled for the size of clinics, specialty, organization type, and region. Independent variables with a P value of less than or equal to 0.05 were deemed to be statistically significant.

RESULTS

The study sites were similar to all CARE Act sites with 2 exceptions (Table 1). The study sites were more likely to be in the South and less likely to be in the Northeast and West, and were less likely to be community health centers.15

TABLE 1
Characteristics all CARE Act Clinics and Study Clinics

A total of 466 monthly activity reports were completed by the 41 study clinics over the 16-month study period (mean, 11.1 reports per clinic). Most (76.2%) of the sites submitted activity reports for at least 8 of the 16 months of the collaborative. On average, clinics attempted 34.7 unique interventions (median 34.5, range, 1–77) (Table 2). The mean breadth index was 0.87 (maximum = 1) suggesting that many clinics attempted interventions for multiple CCM components. The mean depth index was 0.25 (maximum = 1) indicating that few interventions across clinics spanned all the CCM components. Many interventions were special onetime events such as an HIV testing day or a seminar on chronic diseases at a community meeting; on average clinics repeated only 24.8% of the interventions (standard deviation 12.8%, range, 0%–54.2%) at least once. Clinics evaluated only 16.7% of the attempted interventions (standard deviation 12.5%, range, 0%–58.3%).

TABLE 2
Descriptive Statistics for Intervention Choices

Predictors of Number of Interventions

As hypothesized, the number of interventions attempted by a clinic was significantly associated with openness in the organizational culture (P ≤ 0.01) and QI focus (P ≤ 0.05) (Table 3). Community-based organizations, community health centers, and hospitals attempted a combined average of 0.5 fewer interventions than university medical centers (P ≤ 0.05, P ≤ 0.01, P ≤ 0.05, respectively). The number of interventions was not significantly associated with the presence of multidisciplinary teams, regular measurement of quantifiable goals, or the number of active HIV patients in the clinic.

TABLE 3
Predictors of Number of Interventions

Predictors of Comprehensiveness of Interventions

Clinics with a more open organizational culture (P ≤ 0.01), clinics with multidisciplinary teams (P ≤ 0.01), and clinics that measured quantifiable goals more regularly (P ≤ 0.05) were more likely to implement broader, deeper, and more comprehensive interventions than other clinics (Table 4). We obtained similar results when we used different weights for the breadth and depth indices (data not shown) and in models predicting only the breadth and the depth index separately (Table 4).

TABLE 4
Predictors of Intervention Choices

DISCUSSION

In this study, we analyzed the intervention choices made by clinics that participated in a QIC and assessed whether selected clinic characteristics were associated with the number and comprehensiveness of interventions. A clinic’s focus on QI and openness were related to the number and comprehensiveness of the interventions attempted and the presence of multidisciplinary teams and measuring progress toward QI goals were associated with more comprehensive interventions.

Multidisciplinary teams have been found to be effective for improving quality of chronic care,23,33 but our results suggests that the presence of teams alone is not sufficient to increase the number of interventions attempted. The number of interventions might depend primarily on team effectiveness which in turn might depend on team composition, nature of the task, the context, team leadership and processes, and team culture, many of which were not measured in this study.33,40 In addition, team effectiveness might also be related to the quality and suitability of each intervention separately from the total number of interventions, but we did not have independent assessments of these intervention characteristics.

Research on team learning suggests that a supportive organizational and interpersonal climate facilitate collaborative problem-solving and increases the willingness of employees to engage in trial and error experimentation.22,3436 Aspects of such a climate include facilitative leadership, psychologic safety,32,34 and a culture of openness in the organization. Our finding that clinics with more open culture attempted more interventions is consistent with the research on team learning. That research also predicts that organizations with more open cultures will evaluate a higher proportion of the interventions, but our analysis (not shown) did not find that.

The literature on QI stresses the importance of clearly stating quantifiable improvement goals and then measuring progress towards them.25 We found that measurement of progress towards quantifiable goals is associated with the comprehensiveness of the interventions but not with the number of interventions attempted. This could be because implementing more comprehensive interventions requires communication across multiple departments, which might be facilitated by quantification and measurement. On the other hand, merely attempting multiple interventions does not necessarily require quantifiable goals and measurement of progress towards them.

Comprehensiveness of the attempted interventions was significantly associated with the presence of multidisciplinary teams, an open organizational culture, and measurement of progress toward quantifiable goals. This could be because such an organizational culture provides the multidisciplinary teams with a place to discuss issues that span multiple areas and require inputs from different professions. This finding also suggests a likely mechanism through which multidisciplinary teams might affect quality of care: they might focus on interventions that are more comprehensive rather than just increasing the number of interventions attempted. If confirmed by other studies, this would provide important information on how to increase the likelihood that QICs have more positive effect on quality of care.

There are several limitations of our study. We measured only certain aspects of the clinics’ culture. It is likely that characteristics of the teams other than being multidisciplinary such as attitudes of its members and their influence within the clinic could have affected intervention choices.33 Also, the modest clinic level reliability of our measures may have limited our ability to identify important associations.41 Ideally, the comprehensiveness would be calculated by measuring the number of CCM components targeted by each intervention. However, we did not have that information in our reports. There were only 41 clinics that had matching data on interventions and clinician and administrator surveys. More relationships might have been significant with a larger sample size. The data for the study were obtained only from HIV clinics. These findings might not generalize to improvement efforts focused on other conditions.

Demonstrating that organizational factors influence how clinics participate in QICs is only a first step in understanding the heterogeneity in performance of clinics in collaboratives.15,29,31,32 For example, we do not know the extent to which the specific interventions attempted affected care quality. Recently, a study of the Health Disparities Collaborative did not find any relationship between improvement in quality and intervention choices measured by number and percentage of interventions in each CCM category.20 In the future, it would be important to investigate whether the measures of intervention choices developed here along with other structural variables can explain the heterogeneity in the final outcomes.

Supplementary Material

Appendices

Footnotes

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.lww-medicalcare.com).

REFERENCES

1. Chassin MR. Improving the quality of care. New Engl J Med. 1996;335:1060–1063. [PubMed]
2. Chassin MR, Galvin RW. The National Roundtable on Quality. The urgent need to improve health care quality. JAMA. 1998;280:1000–1005. [PubMed]
3. Institute of Medicine (IOM) Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
4. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. New Engl J Med. 2003;348:2635–2645. [PubMed]
5. Berwick DM. A user’s manual for the IOM’s ‘quality chasm’ report. Health Aff. 2002;21:80–90. [PubMed]
6. Leape LL, Berwick DM. Five years after to err is human. What have we learned? JAMA. 2005;293:2384–2390. [PubMed]
7. Blumenthal D. Quality of care. What is it? New Engl J Med. 1996;335:891–894. [PubMed]
8. Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate the progress. Milbank Q. 1998;76:593–624. [PubMed]
9. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q. 2001;79:281–315. [PubMed]
10. Deming WE. Out of the Crisis. Boston, MA: The MIT Press; 1986.
11. Berwick DM. Continuous quality improvement as an ideal in health care. New Engl J Med. 1989;320:53–56. [PubMed]
12. Wilson T, Berwick DM, Cleary PD. What do collaborative improvement projects do? Experience from seven countries. Jt Comm J Qual Patient Saf. 2003;29:85–93. [PubMed]
13. Wagner EH. The role of patient care teams in chronic disease management. BMJ. 2003;20:569–572. [PMC free article] [PubMed]
14. Chin MH, Cook S, Drum ML, et al. Improving diabetes care in Midwest community health centers with the Health Disparities Collaborative. Diabetes Care. 2004;27:2–8. [Erratum in: Diabetes Care. 2004;27:2099] [PubMed]
15. Landon BE, Wilson IB, McInnes DK, et al. Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: the EQHIV study. Ann Intern Med. 2004;140:887–896. [PubMed]
16. Landon BE, Hicks LS, O’Malley AJ, et al. Improving the management of chronic disease at community health centers. New Engl J Med. 2007;356:921–934. [PubMed]
17. Homer CJ, Forbes P, Horvitz L, et al. Impact of a quality improvement program on care and outcomes for children with asthma. Arch Pediatr Adolesc Med. 2005;159:464–469. [PubMed]
18. Schonlau M, Mangione-Smith R, Chan KS, et al. Evaluation of a quality improvement collaborative in asthma care: does it improve processes and outcomes of care? Ann Fam Med. 2005;3:200–208. [PubMed]
19. Asch S, Mangione C, Broder M, et al. Does participation in a collaborative improve quality of care for diabetes? [Accessed May 28, 2007]; Available at: http://www.rand.org/health/projects/icice/asch_abs.html.
20. Grossman E, Keegan T, Lessler AL, et al. Inside the health disparities collaborative: a detailed exploration of quality improvement at community health centers. Med Care. 2008;46:489–496. [PubMed]
21. Pearson ML, Wu S, Schaefer J, et al. Assessing the implementation of the chronic care model in quality improvement collaboratives. Health Serv Res. 2005;40:978–996. [PMC free article] [PubMed]
22. Shortell SM, O’Brien JL, Carman JM, et al. Assessing the impact of continuous quality improvement/total quality management: concept versus implementation. Health Serv Res. 1995;30:377–401. [PMC free article] [PubMed]
23. Wagner EH, Austin BT, Davis C, et al. Improving chronic illness care: translating evidence into action. Health Aff. 2001;20:64–78. [PubMed]
24. Wagner EH, Austin BT, von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74:511–544. [PubMed]
25. Ovretveit J, Bate P, Cleary PD, et al. Quality collaboratives: lessons from research. Qual Saf Health Care. 2002;11:345–351. [PMC free article] [PubMed]
26. Mills PD, Weeks WB. Characteristics of successful quality improvement teams: lessons from five collaborative projects in the VHA. Jt Comm J Qual Saf. 2004;30:152–162. [PubMed]
27. Mittman BS. Creating the evidence base for quality improvement collaboratives. Ann Intern Med. 2004;140:897–901. [PubMed]
28. Boerstler H, Foster RW, O’ Connor EJ, et al. Implementation of total quality management: conventional wisdom vs. reality. Hosp Health Serv Adm. 1996;41:143–159. [PubMed]
29. Plsek P. Collaborating across organizational boundaries to improve quality of care. Am J Infect Control. 1997;25:85–95. [PubMed]
30. Zammuto RF, Krakower JY. 1999. Quantitative and qualitative studies of organizational culture. In: Pasmore WA, Woodman RW, editors. Research in Organizational Change and Development. Vol. 5. Greenwich, United Kingdom: JAI Press; 1999. pp. 83–114.
31. Shortell SM, Jones RH, Rademaker AW, et al. Assessing the impact of total quality management and organization culture on multiple outcomes of care for coronary artery bypass graft surgery patients. Med Care. 2000;38:207–217. [PubMed]
32. Shortell SM, Zazzali JL, Burns LR, et al. Implementing evidence-based medicine: the role of market pressures, compensation incentives, and culture in physician organizations. Med Care. 2001;39:62–78. [PubMed]
33. Shortell SM, Marsteller JA, Lin M, et al. The role of perceived team effectiveness in improving chronic illness care. Med Care. 2004;42:1040–1048. [PubMed]
34. Edmondson AC, Bohmer R, Pisano GP. Disrupted routines: team learning and new technology adaptation. Admin Sci Quart. 2001;46:685–716.
35. Sarin S, McDermott C. Effect of team leader characteristics on learning, knowledge application, and performance of cross-functional new product development teams. Decision Sci. 2003;34:707–739.
36. Edmondson AC. Psychological safety and learning behavior in work teams. Admin Sci Quart. 1999;44:350–383.
37. Argote L, Gruenfeld D, Naquin C. Group learning in organizations. In: Turner ME, editor. Groups at Work: Advances in Theory and Research. New York, NY: Erlbaum; 2001. pp. 369–411.
38. Horbar JD, Rogowski J, Plesk P, et al. Collaborative quality improvement for neonatal intensive care. Pediatrics. 2001;107:14–22. [PubMed]
39. Cameron AC, Trivedi PK. Regression Analysis of Count Data. Cambridge: Cambridge University Press; 1998.
40. Fried B, Topping S, Edmondson A. Groups and Teams, Chapt 6. In: Shortell SM, Kaluzny AD, editors. Health Care Management: Organization Design and Behavior. 5th ed. Berkely, CA: University of California; 2006.
41. Marsden PV, Landon BE, Wilson IB, et al. The reliability of survey assessments of characteristics of medical clinics. Health Serv Res. 2006;41:265–283. [PMC free article] [PubMed]
42. Scherer FM. Industrial Market Structure and Economic Performance. Chicago, IL: Rand McNally; 1970.
43. Baumgardner JR, Marder WD. Specialization among obstetrician gynecologists-Another dimension of physician supply. Med Care. 1991;29:272–282. [PubMed]
44. Franks P, Williams GC, Zwanziger J, et al. Why do physicians vary so widely in their referral rates? J Gen Intern Med. 2000;15:163–168. [PMC free article] [PubMed]
45. Lescoe-Long MA, Long MJ, Amidon RL, et al. The relationship between resource constraints and physician problem solving-implications for improving the process of care. Med Care. 1996;34:931–953. [PubMed]
46. Nunnally JC. Psychometric Theory. New York, NY: McGraw-Hill; 1967.