|Home | About | Journals | Submit | Contact Us | Français|
Complexity science suggests that our current health care delivery system acts as a complex adaptive system (CAS). Such systems represent a dynamic and flexible network of individuals who can coevolve with their ever changing environment. The CAS performance fluctuates and its members’ interactions continuously change over time in response to the stress generated by its surrounding environment. This paper will review the challenges of intervening and introducing a planned change into a complex adaptive health care delivery system. We explore the role of the “reflective adaptive process” in developing delivery interventions and suggest different evaluation methodologies to study the impact of such interventions on the performance of the entire system. We finally describe the implementation of a new program, the Aging Brain Care Medical Home as a case study of our proposed evaluation process.
In 2001, the Institute of Medicine (IOM) identified suboptimal health care quality, compromised patient safety, and waste in our current health care system.1 These quality issues are found among a wide range of chronic conditions, including dementia. In 2005, Americans spent an estimated $100 billion providing care for three million Americans and their family caregivers affected by dementia.2,3 By 2011, these costs are expected to double.4 Despite the high cost of this care among those with recognized dementia, only 20% to 30% of Americans with dementia are diagnosed with the condition5,6 and multiple other quality problems have been reported. For example, 50% of older adults with dementia are exposed to a potentially inappropriate medication; only 7% are prescribed cholinesterase inhibitors; more than 20% are prescribed neuroleptics that have not been approved for older adults with dementia; and 33% utilize acute care services at least once every six months.5–7
The IOM recommended new organizational approaches and systems frameworks to ensure that the 21st Century American health care system delivers safe, effective, patient-centered, timely, efficient, and equitable care.1 In this white paper, we use complex adaptive system (CAS) theory to provide the reader with a framework to select, implement, and evaluate new care delivery models that have the potential of fulfilling the IOM recommendations.
A CAS is a dynamic network of semiautonomous, competing, and collaborating individuals who interact and coevolve in nonlinear ways with their surrounding environment.8–11 These interactions lead to various webs of relationships that influence the system’s performance.8–12 By receiving and storing lessons from previous experiences and through modification of the relationships among its members, a CAS displays emergent self-organized behaviors and horizontal controls8,9,11,13–15 (Figure 1).
Health care delivery organizations are considered complex adaptive systems.7,10,13,16–24 These organizations are composed of semiautonomous individuals who interact constantly in a nonlinear way while faced with external and internal stressors such as patients’ medical status, insurance requirements, regulations, new research findings, members’ turnover, and legal issues.7,10,13,16–24 Traditional conceptual models of the health care delivery system often portray the health care system as a machine with replaceable parts and predictable behaviors that can be changed and reproduced based on past performance data.10,13,18,21–26 This view assumes that stability is the natural state of these organizations; that they consist of functions and roles that are carried out by replaceable nurses and physicians; and that financial incentives and regulatory policies offer recipes for predictable improvements in the performance.10,13,18,21–26
However, a new wave of health care leaders argue that the assembly line conceptual model does not fit health care systems.10,11,13,15,18,21,22,24–28 Rather, they conceptualize their organizations as a CAS with local critical nonlinear relationships that produce unpredictable behavioral patterns.10,11,13,15,18,21,22,24–28 Attempts to rigidly control these CASs often worsen the targeted problems and lead to unintended negative consequences.10,11,13,15,18,21,22,24–31
A health care leader using the lens of CAS understands that the capability of the health care delivery organization to adapt to an ever-changing environment depends on the skills and adaptability of its individual employees, their relationships and interactions, and the organizational communication patterns with other organizations.8,10,11,13,18,19,32
The research to support such an innovative approach is already beginning to emerge.20,33 In a systematic evidence review of 32 intervention studies seeking to improve chronic diabetes mellitus care, Leykum et al found a positive linear association between the success of a specific delivery intervention and the use of a CAS theory to develop or implement the intervention.20 In another systematic review of factors affecting the success of various programs to disseminate innovations in health service delivery, Greenhalgh found that some of the factors influencing adoption of innovations were: 1) the adaptability of the innovation to the needs of the adopter; 2) the feasibility, workability, and ease of use of the innovation; 3) the harnessing of the influence of true opinion leaders; and 4) the support of informal interorganizational networks for the adoption of the innovations.33 These factors resemble the framework and the characteristics of a CAS.
While the CAS framework is attractive for understanding the aspects of a health care organization, the method by which the CAS framework is used to implement an intervention is not explicit. Crabtree et al developed the “Reflective Adaptive Process” (RAP)13,18,24,26,34 to apply complexity science principles to select, develop, and implement a change in health care delivery systems.13,18,24,26,34 The RAP was developed over a series of large descriptive and intervention studies of primary care practices funded by the National Institutes of Health (NIH) and the Agency for Healthcare Research and Quality.13,18,24,26,34 RAP has five guiding principles that offer focus to implementation efforts without prescribing specific actions.13,18,24,26,34
The RAP starts with forming a cross-functional team that begins to meet regularly. This RAP team uses iterative cycles to identify priority improvement opportunities, discuss potential solutions, pilot several changes, and reflect on the impact of changes. A facilitator helps the team develop the necessary skills of group process, conflict management, meeting management, team building, and reflection-action cycles.
The RAP is considered a new Quality Improvement (QI) method that compliments the previous success of the other QI initiatives by recognizing the interdependence of system members and allowing the system leaders to create an optimal matrix for both the system members and their surrounding environment to coevolve.13,18,24,26,34,35
The effectiveness of RAP has been evaluated by the National Heart, Lung, and Blood Institute (NHLBI)-funded ULTRA (Using Learning Teams for Reflective Adaptation) study. ULTRA is a five-year group randomized clinic trial of 60 primary care practices in New Jersey and Pennsylvania that use RAP to enhance the care delivery for multiple chronic conditions.26,34 We have used RAP to develop a computerized decision support system for hospitalized older adults with cognitive impairment that was funded by the National Institute on Aging (K23-AG-26770–01).16,17 We also used the RAP principles to successfully build the Aging Brain Care Medical Home (ABC-MedHome) as described in detail later.
Currently, the process of approving a drug, device or new care delivery model requires a randomized double blind controlled clinical trial (RCT) to produce unbiased evidence of efficacy or safety.36–39 Without the use of RCT design, evaluating the efficacy of any change would be susceptible to various forms of bias such as the placebo response, the effects of unknown confounders, the natural history of any acute or chronic illness, and patients’ and providers’ negative or positive expectations of any introduced intervention.38 At the same time, extrapolating the results produced by a RCT suffers from generalizability limitations at the local implementation phase.36–40 Such generalizability limitations are due to the local and unique characteristics of the complex adaptive health care delivery system and demand local modification of the planned change.27,38,39 Ironically, the efficacy of such a modified locally sensitive version of the intervention would technically need to be tested in a new RCT. This paradoxical cycle is the main source of challenge in evaluation within the complex adaptive health care delivery systems. We need a solution that balances the internal and the external validity of the experimental intervention with the realities of a CAS.
We suggest two evaluation designs to test an intervention introduced into a specific CAS (Table 1): the use of N-of-1 randomized controlled trial (N-of-1 RCT);14,41–43 or a standardized pre–post time series design.
Guyatt et al introduced N-of-1 RCTs to medicine in 1986.41 Essentially the N-of-1 RCT uses time series data for both independent and dependent variables to evaluate the effect of therapy on one person. N-of-1 RCTs consist of a random sequence of different interventions that may include placebo, usual care or any other control, administered in double-blind protocol with regular and standardized measurement of specific intervention effects (efficacy measure) and measurement of impact on the entire system (harmful or unexpected positive or negative impact on the entire system).14,41–43 Typical group-based RCTs inform us of the average magnitude of an effect in a group of CASs. However, health administrators, managing a specific CAS, are concerned with knowing what is going to happen in their own health care delivery system.10,23
After selecting a planned change and choosing N-of-1-RCT as the evaluation design, three sets of outcomes need to be monitored. The first outcome set monitors the performance of the entire complex adaptive health care delivery system such as overall quality, cost, clinicians’ satisfactions, overall occupancy, overall errors, and overall length of stay. The second set is specific to the direct impact of the selected intervention. The third set monitors and measures the processes that mediate the effect of the interventions.14,41–43 The statistical analysis can be the application of a “sign test” that is based on the binomial distribution; and assuming there is no treatment difference, the probability of three consecutive pairs of treatment periods favoring active treatment is one in eight or 0.125 (one-sided test).14,41–43
In cases where the use of N-of-1 RCT is not possible then an alternative option could be a standardized pre-post time series analysis. This design includes the following elements:
We conclude this paper by providing the reader with an example of a quality improvement project that follows the steps that we discussed regarding selecting an intervention in a complex adaptive health care delivery system and evaluating its impact (Table 2).
Although primary care practice does not have the resources to provide the appropriate assessment and the complex management required for patients with dementia and depression,5,6,7 redesigning this practice setting to accommodate the special needs of these patients and their caregivers can improve health outcomes as evidenced by recent randomized controlled trials.12,44–46
Using the RAP method, the Indiana University Aging Brain program adapted the collaborative care model into the locally sensitive ABC-MedHome program. The goal of this program is to find, assess, and manage the biospsychosocial needs of older patients suffering from dementia or depression, and their family caregivers who are receiving care within the primary care practice affiliated with Wishard Health Services (WHS).
WHS is an urban integrated public health care system serving residents of Marion County in Indiana. Guided by the RAP method, the implementation team was provided with the necessary time, space, semiautonomous decision making capacity, diversity, and leadership support to work with the clinical providers involved in the care of older adults with dementia or depression to agree on a locally acceptable approach to implementing the collaborative care model (Table 2). The team was provided with the existing guidelines and protocols from the Collaborative Care studies to develop a locally sensitive and minimally standardized care approach including consensus on the specific choice of screening instruments, diagnostic criteria, and measures of severity that will be used to deliver care at ABC-MedHome. The team also specified the role of the ABC-MedHome coordinator and director and other providers, the criteria for co-management, and the criteria for referral to memory care specialists. Finally, the team attended to the need to integrate current guidelines for care among those patients who suffer from dementia and depression with particular attention to the co-management of cardiovascular diseases. Although caregivers are targeted as a vehicle to deliver care to the care recipient, the team developed a standardized approach to directly detecting and treating the caregiver for depression.
To accomplish the above goals, the team met on a weekly basis for approximately one year and worked closely via quarterly reports with the leadership of the target health care system to understand the opportunities and barriers to the implementation of the ABC-MedHome from a financial, human resource, and space perspective.
As mentioned earlier, implementation of change in a health care system often fails because organizers fail to recognize the complex adaptive system properties of this dynamic human system and design an inappropriate feedback process that monitors the impact of the change on the entire system. Using the conceptual model of CAS and the resources from National Institute of Mental Health (P30AG024967), we developed an evaluation platform that would provide continuous and timely data relevant to the performance of the ABC-MedHome for the leadership of WHS and would also support conducting time series analyses that could provide an adequate method to investigate any potential causal association between the ABC-MedHome and certain system outcomes.
The evaluation matrix included the following specifications:
Based on the data from the three clinical trials that evaluated the impact of the collaborative care model, using the above criteria, and utilizing the capacity of the local health care system electronic medical record system, the implementation team has finalized the ABC-MedHome performance platform (See Table 3).
The program was developed and is currently undergoing field-testing at WHS. There are three characteristic of WHS that might limit the generalizability of the ABC-MedHome. First, WHS provides care to a diverse urban and vulnerable population of older adults many of whom are low-income and subject to multiple socioeconomic stressors. Thus, generalizability to rural and solo primary care practices is limited. Second, WHS has been the site of the two randomized trials of the collaborative care model for both dementia and depression and thus, the buy-in of the leadership and the clinicians of the locally sensitive content of the ABC-MedHome had already been facilitated. Third, WHS is served by one of the nation’s oldest and most comprehensive electronic medical records that captures diagnostic studies, physician orders, drug dispensing, death certificates, encounter information, dictated reports, hospital admission and discharge diagnoses, dates and lengths of hospital stay, and discharge disposition. Thus, health care systems that do not have similar electronic systems may struggle in providing data for the evaluation platform of the ABC-MedHome. However, some of these features are shared by other health care systems. For example, the replication of ABC-MedHome is possible in systems such as the Veteran Affairs (VA) health care system and Kaiser Permanente health system. Both have a comprehensive electronic medical records system, are considered integrated health care systems, and have conducted numerous research activities related to the collaborative care model. Indeed both Kaiser and the VA were performance sites for the collaborative care model RCTs.44,45
The current research infrastructure and the CAS nature of our health care delivery organizations are not well designed to support the translational “discovery to delivery” step that will be required to reduce the current and future burden of chronic care epidemics facing our American society. The IOM and the NIH recognized this large gap in translating research innovations from discovery to delivery and recommended urgent “re-engineering of the clinical research enterprise”. Understanding our health care delivery system as a CAS might transform our health care system into a new integrated adaptive discovery network capable of continuously, instantly and efficiently improving the safety, quality, and cost utility of the American health care system. Such an understanding may enhance our capacity to bridge research findings and the daily dynamics of these organizations and thus develop a flexible and local process of a minimally standardized approach with continuous feedback loops to maintain the best probability of surviving the often unpredictable future challenges that will face these health care delivery systems.
Dr Boustani is supported by the National Institute on Aging (NIA) Paul B. Beeson K23 Career Development Award # 1-K23-AG026770-01 and R01AG029884-01. Dr Callahan is supported by NIA awards K24-AG026770 and P30AG024967. There are no conflicts of interest for these authors.