This paper describes initial testing of an instrument to evaluate key elements of structure and processes of care for chronic illness. Our results and feedback from teams suggest that the ACIC is responsive to changes resulting from quality improvement efforts in health-care settings. Baseline scores were generally similar across teams addressing different chronic illnesses, and consistently showed improvement after intervention across Chronic Care Model elements. Moreover, the ACIC correlates positively with ratings of teams' performance outcomes by faculty experts leading the quality improvement collaboratives. Feedback from participating teams suggests that they find the ACIC extremely useful for identifying areas in which they need to focus improvement efforts, and in tracking progress over time. Initial experience suggests that the ACIC is very applicable across different types of health-care systems (e.g., for profit, IPA, community health centers, hospital-based programs) and chronic illnesses. The collaboratives asked teams to focus their activities on a small number of practices (e.g., 1–3), which was the focus for the response to the ACIC.
The ACIC typically requires 15–30 minutes to complete including time spent reaching agreement on ratings. Discussions of areas and specific improvement strategies are prompted by the item anchors provided and follow naturally from completion of the ratings. These discussions typically assist teams in identifying areas for improvement for chronic illness care. In most cases, improvement in all six areas of the ACIC are needed, including organization of care, establishing formal links to community resources, developing formal supports for self-management, decision support, delivery system design, and information systems. One of the advantages of the ACIC is that the most advanced category (the highest possible score for each item) describes optimal practice. Thus, teams have guidance as to what comprises the best care for chronic illness. To further assist teams in translating their scores into practical terms for focusing their improvement efforts, we are developing and formally testing a feedback form that will be used in future chronic care collaboratives.
There are limitations to the evidence presented to date. The ACIC was developed to supplement our quality-improvement work with health-care delivery systems. Thus, it was not subject to the typical instrument development process involving a complete review of the literature to identify content for items. The development of the Chronic Care Model, the conceptual framework for the ACIC, has been described in a previous article (Wagner et al. 1999
). It was derived from a combination of surveys of best practices, quality improvement activities, expert opinion, and consistency with the more promising interventions in the literature. It was not intended to be a representation of available evidence, but a heuristic, practical tool that would be changed in light of new evidence. We intend to add evidence for the domains of the Chronic Care Model and ACIC in a formal literature review as it becomes available. For example, although there are limited research studies that examine the impact of leadership on practice redesign in chronic illness care, there is evidence in other industries that leadership support is important in making organizational changes. We will continue to monitor the literature on studies that address this issue in relation to chronic illness care improvement.
The analyses on the ACIC were undertaken as a secondary objective of our quality improvement work, rather than as part of a formal research study. Thus, we did not impose a formal study design or select teams to be involved in the collaboratives. The involved organizations were relatively small in number and highly motivated, which may not be representative of the average health care organization. We would like to have had complete data from all teams to more fully depict improvement across the collaboratives. Future chronic care collaboratives will involve a more rigorous data collection strategy to minimize missing data. Moreover, additional data need to be collected by other research groups to further document the reliability and validity of the ACIC.
Although the ACIC was responsive to improvement efforts, the presence of a control group (or control “sites”) would have strengthened the conclusions. While it is possible that simply completing the ACIC could act as an intervention itself based on the “education” teams receive in completing the survey, we do not think it likely given the difficulty in producing organizational change. Nevertheless, the RAND–UC-Berkeley evaluation of the collaboratives will include a formal comparison of enrolled teams to “control groups,” that is, clinics not actively participating in the quality improvement effort, to determine whether differences in process measures (e.g., ACIC scores) and outcomes exist. At this writing, it was not possible to evaluate whether changes in the ACIC represent verifiable changes teams made in their systems, or furthermore, whether teams focused organizational change on all six aspects of the Chronic Care Model. As part of the official evaluation of the collaboratives, each of the interventions (“PDSA” cycles) teams make in their system will be coded according to Chronic Care Model elements. This will allow us to compare whether changes on the ACIC are related to the number and/or intensity of changes teams make in different areas, or whether certain aspects of the Chronic Care Model—e.g., self-management support—appear to be more important in improving outcomes.
In conclusion, preliminary data suggest that the ACIC is a useful quality improvement tool. While additional research is clearly indicated, the instrument appears sensitive to intervention changes across different chronic illnesses and helps teams focus their efforts on adopting evidence-based chronic care changes.