Search tips
Search criteria 


Logo of clinintagDove Medical PressSubscribeSubmit a ManuscriptSearchFollowDovepressClinical Interventions in Aging
Clin Interv Aging. 2010; 5: 141–148.
Published online 2010 May 25.
PMCID: PMC2877524

Selecting a change and evaluating its impact on the performance of a complex adaptive health care delivery system


Complexity science suggests that our current health care delivery system acts as a complex adaptive system (CAS). Such systems represent a dynamic and flexible network of individuals who can coevolve with their ever changing environment. The CAS performance fluctuates and its members’ interactions continuously change over time in response to the stress generated by its surrounding environment. This paper will review the challenges of intervening and introducing a planned change into a complex adaptive health care delivery system. We explore the role of the “reflective adaptive process” in developing delivery interventions and suggest different evaluation methodologies to study the impact of such interventions on the performance of the entire system. We finally describe the implementation of a new program, the Aging Brain Care Medical Home as a case study of our proposed evaluation process.

Keywords: complexity, aging brain, implementation, complex adaptive system, sustained change, care delivery


In 2001, the Institute of Medicine (IOM) identified suboptimal health care quality, compromised patient safety, and waste in our current health care system.1 These quality issues are found among a wide range of chronic conditions, including dementia. In 2005, Americans spent an estimated $100 billion providing care for three million Americans and their family caregivers affected by dementia.2,3 By 2011, these costs are expected to double.4 Despite the high cost of this care among those with recognized dementia, only 20% to 30% of Americans with dementia are diagnosed with the condition5,6 and multiple other quality problems have been reported. For example, 50% of older adults with dementia are exposed to a potentially inappropriate medication; only 7% are prescribed cholinesterase inhibitors; more than 20% are prescribed neuroleptics that have not been approved for older adults with dementia; and 33% utilize acute care services at least once every six months.57

The IOM recommended new organizational approaches and systems frameworks to ensure that the 21st Century American health care system delivers safe, effective, patient-centered, timely, efficient, and equitable care.1 In this white paper, we use complex adaptive system (CAS) theory to provide the reader with a framework to select, implement, and evaluate new care delivery models that have the potential of fulfilling the IOM recommendations.

The health care delivery system as a complex adaptive system

A CAS is a dynamic network of semiautonomous, competing, and collaborating individuals who interact and coevolve in nonlinear ways with their surrounding environment.811 These interactions lead to various webs of relationships that influence the system’s performance.812 By receiving and storing lessons from previous experiences and through modification of the relationships among its members, a CAS displays emergent self-organized behaviors and horizontal controls8,9,11,1315 (Figure 1).

Figure 1
The complex adaptive health care system.

Health care delivery organizations are considered complex adaptive systems.7,10,13,1624 These organizations are composed of semiautonomous individuals who interact constantly in a nonlinear way while faced with external and internal stressors such as patients’ medical status, insurance requirements, regulations, new research findings, members’ turnover, and legal issues.7,10,13,1624 Traditional conceptual models of the health care delivery system often portray the health care system as a machine with replaceable parts and predictable behaviors that can be changed and reproduced based on past performance data.10,13,18,2126 This view assumes that stability is the natural state of these organizations; that they consist of functions and roles that are carried out by replaceable nurses and physicians; and that financial incentives and regulatory policies offer recipes for predictable improvements in the performance.10,13,18,2126

However, a new wave of health care leaders argue that the assembly line conceptual model does not fit health care systems.10,11,13,15,18,21,22,2428 Rather, they conceptualize their organizations as a CAS with local critical nonlinear relationships that produce unpredictable behavioral patterns.10,11,13,15,18,21,22,2428 Attempts to rigidly control these CASs often worsen the targeted problems and lead to unintended negative consequences.10,11,13,15,18,21,22,2431

The reflective adaptive process of selecting and implementing a change in a complex adaptive health care delivery system

A health care leader using the lens of CAS understands that the capability of the health care delivery organization to adapt to an ever-changing environment depends on the skills and adaptability of its individual employees, their relationships and interactions, and the organizational communication patterns with other organizations.8,10,11,13,18,19,32

The research to support such an innovative approach is already beginning to emerge.20,33 In a systematic evidence review of 32 intervention studies seeking to improve chronic diabetes mellitus care, Leykum et al found a positive linear association between the success of a specific delivery intervention and the use of a CAS theory to develop or implement the intervention.20 In another systematic review of factors affecting the success of various programs to disseminate innovations in health service delivery, Greenhalgh found that some of the factors influencing adoption of innovations were: 1) the adaptability of the innovation to the needs of the adopter; 2) the feasibility, workability, and ease of use of the innovation; 3) the harnessing of the influence of true opinion leaders; and 4) the support of informal interorganizational networks for the adoption of the innovations.33 These factors resemble the framework and the characteristics of a CAS.

While the CAS framework is attractive for understanding the aspects of a health care organization, the method by which the CAS framework is used to implement an intervention is not explicit. Crabtree et al developed the “Reflective Adaptive Process” (RAP)13,18,24,26,34 to apply complexity science principles to select, develop, and implement a change in health care delivery systems.13,18,24,26,34 The RAP was developed over a series of large descriptive and intervention studies of primary care practices funded by the National Institutes of Health (NIH) and the Agency for Healthcare Research and Quality.13,18,24,26,34 RAP has five guiding principles that offer focus to implementation efforts without prescribing specific actions.13,18,24,26,34

  1. Vision, mission, and shared values are fundamental in guiding ongoing change processes”.
  2. “Creating time and space for learning and reflection is necessary”.
  3. Tension and discomfort are essential and normal during change”.
  4. “Improvement teams should include a variety of systems agents with different perspectives of the system and its environment, including patients”.
  5. “System change requires supportive leadership that is actively involved in the change process, ensuring full participation from all members and protecting time for reflection”.13,18,24,26,34

The RAP starts with forming a cross-functional team that begins to meet regularly. This RAP team uses iterative cycles to identify priority improvement opportunities, discuss potential solutions, pilot several changes, and reflect on the impact of changes. A facilitator helps the team develop the necessary skills of group process, conflict management, meeting management, team building, and reflection-action cycles.

The RAP is considered a new Quality Improvement (QI) method that compliments the previous success of the other QI initiatives by recognizing the interdependence of system members and allowing the system leaders to create an optimal matrix for both the system members and their surrounding environment to coevolve.13,18,24,26,34,35

The effectiveness of RAP has been evaluated by the National Heart, Lung, and Blood Institute (NHLBI)-funded ULTRA (Using Learning Teams for Reflective Adaptation) study. ULTRA is a five-year group randomized clinic trial of 60 primary care practices in New Jersey and Pennsylvania that use RAP to enhance the care delivery for multiple chronic conditions.26,34 We have used RAP to develop a computerized decision support system for hospitalized older adults with cognitive impairment that was funded by the National Institute on Aging (K23-AG-26770–01).16,17 We also used the RAP principles to successfully build the Aging Brain Care Medical Home (ABC-MedHome) as described in detail later.

Evaluating the impact of a planned change on the performance of a complex adaptive health care delivery system

Currently, the process of approving a drug, device or new care delivery model requires a randomized double blind controlled clinical trial (RCT) to produce unbiased evidence of efficacy or safety.3639 Without the use of RCT design, evaluating the efficacy of any change would be susceptible to various forms of bias such as the placebo response, the effects of unknown confounders, the natural history of any acute or chronic illness, and patients’ and providers’ negative or positive expectations of any introduced intervention.38 At the same time, extrapolating the results produced by a RCT suffers from generalizability limitations at the local implementation phase.3640 Such generalizability limitations are due to the local and unique characteristics of the complex adaptive health care delivery system and demand local modification of the planned change.27,38,39 Ironically, the efficacy of such a modified locally sensitive version of the intervention would technically need to be tested in a new RCT. This paradoxical cycle is the main source of challenge in evaluation within the complex adaptive health care delivery systems. We need a solution that balances the internal and the external validity of the experimental intervention with the realities of a CAS.

We suggest two evaluation designs to test an intervention introduced into a specific CAS (Table 1): the use of N-of-1 randomized controlled trial (N-of-1 RCT);14,4143 or a standardized pre–post time series design.

Table 1
Selecting a change in a complex adaptive health care delivery system

The N-of-1 RCT design

Guyatt et al introduced N-of-1 RCTs to medicine in 1986.41 Essentially the N-of-1 RCT uses time series data for both independent and dependent variables to evaluate the effect of therapy on one person. N-of-1 RCTs consist of a random sequence of different interventions that may include placebo, usual care or any other control, administered in double-blind protocol with regular and standardized measurement of specific intervention effects (efficacy measure) and measurement of impact on the entire system (harmful or unexpected positive or negative impact on the entire system).14,4143 Typical group-based RCTs inform us of the average magnitude of an effect in a group of CASs. However, health administrators, managing a specific CAS, are concerned with knowing what is going to happen in their own health care delivery system.10,23

After selecting a planned change and choosing N-of-1-RCT as the evaluation design, three sets of outcomes need to be monitored. The first outcome set monitors the performance of the entire complex adaptive health care delivery system such as overall quality, cost, clinicians’ satisfactions, overall occupancy, overall errors, and overall length of stay. The second set is specific to the direct impact of the selected intervention. The third set monitors and measures the processes that mediate the effect of the interventions.14,4143 The statistical analysis can be the application of a “sign test” that is based on the binomial distribution; and assuming there is no treatment difference, the probability of three consecutive pairs of treatment periods favoring active treatment is one in eight or 0.125 (one-sided test).14,4143

N-of-1 RCT might be appropriate when several of the following criteria coexist:14,4143

  1. There are significant doubts about an intervention’s effectiveness in a specific complex adaptive health care delivery system;
  2. The intervention might have potential adverse effects on the complex adaptive health care delivery system;
  3. The selected intervention is expensive and its impact needs to be confirmed to justify these expenses;
  4. The intervention is targeting a chronic but stable poor performance problem within a specific complex adaptive health care delivery system;
  5. The evaluation team has the capacity to monitor all three types of outcomes mentioned above.

A Standard Pre-Post Time Series design

In cases where the use of N-of-1 RCT is not possible then an alternative option could be a standardized pre-post time series analysis. This design includes the following elements:

  1. Selection of specific outcomes that are targeted by the intervention and choose a reliable measurement method;
  2. Measuring the performance of the entire system using already existing data such as overall cost, overall quality, admission rates, death rate, discharge rate, patient satisfaction, and other data;
  3. Select a preintervention and intervention period to measure the system and intervention-related outcomes.

A case study, the aging brain care medical home (ABC-MedHome) program

We conclude this paper by providing the reader with an example of a quality improvement project that follows the steps that we discussed regarding selecting an intervention in a complex adaptive health care delivery system and evaluating its impact (Table 2).

Table 2
Using the RAP to translate the collaborative care model into the ABC-MedHome

Although primary care practice does not have the resources to provide the appropriate assessment and the complex management required for patients with dementia and depression,5,6,7 redesigning this practice setting to accommodate the special needs of these patients and their caregivers can improve health outcomes as evidenced by recent randomized controlled trials.12,4446

Using the RAP method, the Indiana University Aging Brain program adapted the collaborative care model into the locally sensitive ABC-MedHome program. The goal of this program is to find, assess, and manage the biospsychosocial needs of older patients suffering from dementia or depression, and their family caregivers who are receiving care within the primary care practice affiliated with Wishard Health Services (WHS).

Translating the collaborative care model into the ABC-MedHome

WHS is an urban integrated public health care system serving residents of Marion County in Indiana. Guided by the RAP method, the implementation team was provided with the necessary time, space, semiautonomous decision making capacity, diversity, and leadership support to work with the clinical providers involved in the care of older adults with dementia or depression to agree on a locally acceptable approach to implementing the collaborative care model (Table 2). The team was provided with the existing guidelines and protocols from the Collaborative Care studies to develop a locally sensitive and minimally standardized care approach including consensus on the specific choice of screening instruments, diagnostic criteria, and measures of severity that will be used to deliver care at ABC-MedHome. The team also specified the role of the ABC-MedHome coordinator and director and other providers, the criteria for co-management, and the criteria for referral to memory care specialists. Finally, the team attended to the need to integrate current guidelines for care among those patients who suffer from dementia and depression with particular attention to the co-management of cardiovascular diseases. Although caregivers are targeted as a vehicle to deliver care to the care recipient, the team developed a standardized approach to directly detecting and treating the caregiver for depression.

To accomplish the above goals, the team met on a weekly basis for approximately one year and worked closely via quarterly reports with the leadership of the target health care system to understand the opportunities and barriers to the implementation of the ABC-MedHome from a financial, human resource, and space perspective.

Planned evaluation of The ABC-MedHome

As mentioned earlier, implementation of change in a health care system often fails because organizers fail to recognize the complex adaptive system properties of this dynamic human system and design an inappropriate feedback process that monitors the impact of the change on the entire system. Using the conceptual model of CAS and the resources from National Institute of Mental Health (P30AG024967), we developed an evaluation platform that would provide continuous and timely data relevant to the performance of the ABC-MedHome for the leadership of WHS and would also support conducting time series analyses that could provide an adequate method to investigate any potential causal association between the ABC-MedHome and certain system outcomes.

The evaluation matrix included the following specifications:

  1. Define the borders of the complex adaptive system by selecting the patients who would potentially interact in a direct or indirect way with the program.
  2. Determine and measure the outcomes of the entire system using already existing data such as services utilization data, quality care, admission rate, death rate, discharge rate, patient satisfaction, and other data.
  3. Select preintervention and postintervention periods to measure the above outcomes.
  4. Select the length and the frequency of performance reports.

Based on the data from the three clinical trials that evaluated the impact of the collaborative care model, using the above criteria, and utilizing the capacity of the local health care system electronic medical record system, the implementation team has finalized the ABC-MedHome performance platform (See Table 3).

Table 3
The evaluation platform or the matrix of the ABC-MedHome performance

The generalizability of the Abc-Medhome

The program was developed and is currently undergoing field-testing at WHS. There are three characteristic of WHS that might limit the generalizability of the ABC-MedHome. First, WHS provides care to a diverse urban and vulnerable population of older adults many of whom are low-income and subject to multiple socioeconomic stressors. Thus, generalizability to rural and solo primary care practices is limited. Second, WHS has been the site of the two randomized trials of the collaborative care model for both dementia and depression and thus, the buy-in of the leadership and the clinicians of the locally sensitive content of the ABC-MedHome had already been facilitated. Third, WHS is served by one of the nation’s oldest and most comprehensive electronic medical records that captures diagnostic studies, physician orders, drug dispensing, death certificates, encounter information, dictated reports, hospital admission and discharge diagnoses, dates and lengths of hospital stay, and discharge disposition. Thus, health care systems that do not have similar electronic systems may struggle in providing data for the evaluation platform of the ABC-MedHome. However, some of these features are shared by other health care systems. For example, the replication of ABC-MedHome is possible in systems such as the Veteran Affairs (VA) health care system and Kaiser Permanente health system. Both have a comprehensive electronic medical records system, are considered integrated health care systems, and have conducted numerous research activities related to the collaborative care model. Indeed both Kaiser and the VA were performance sites for the collaborative care model RCTs.44,45


The current research infrastructure and the CAS nature of our health care delivery organizations are not well designed to support the translational “discovery to delivery” step that will be required to reduce the current and future burden of chronic care epidemics facing our American society. The IOM and the NIH recognized this large gap in translating research innovations from discovery to delivery and recommended urgent “re-engineering of the clinical research enterprise”. Understanding our health care delivery system as a CAS might transform our health care system into a new integrated adaptive discovery network capable of continuously, instantly and efficiently improving the safety, quality, and cost utility of the American health care system. Such an understanding may enhance our capacity to bridge research findings and the daily dynamics of these organizations and thus develop a flexible and local process of a minimally standardized approach with continuous feedback loops to maintain the best probability of surviving the often unpredictable future challenges that will face these health care delivery systems.


Dr Boustani is supported by the National Institute on Aging (NIA) Paul B. Beeson K23 Career Development Award # 1-K23-AG026770-01 and R01AG029884-01. Dr Callahan is supported by NIA awards K24-AG026770 and P30AG024967. There are no conflicts of interest for these authors.


1. Institute of Medicine . Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001. [PubMed]
2. Sloane PD, et al. The public health impact of Alzheimer’s disease, 2000–2050: potential implication of treatment advances. Ann Rev Public Health. 2002;23:213–231. [PubMed]
3. Wimo A, Winblad B, Jonsson L. An estimate of the total worldwide societal costs of dementia in 2005. Alzheimers Dement. 2007 Apr;3(2):81–91. [PubMed]
4. Alzheimer Association [Internet page]. Alzheimer’s disease facts and figures. Available from: Accessed 2007 April 24.
5. Boustani M, et al. Implementing a screening and diagnosis program for dementia in primary care. J Gen Intern Med. 2005 Jul;20(7):572–577. [PMC free article] [PubMed]
6. Boustani M, Peterson B, Hanson L, Harris R, Lohr KN, US Preventive Task Force Screening for dementia in primary care: a summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2003 Jun 3;138(11):927–937. [PubMed]
7. Boustani M, Schubert C, Sennour Y. The challenge of supporting care for dementia in primary care. Clin Interv Aging. 2007;2(4):631–636. [PMC free article] [PubMed]
8. Holden LM. Complex adaptive systems: concept analysis. J Adv Nurs. 2005 Dec;52(6):651–657. [PubMed]
9. Lansing SJ. Complex adaptive systems. Annu Rev Anthropol. 2003;32:183–204.
10. Matthews JI, Thomas PT. Managing clinical failure: a complex adaptive system perspective. Int J Health Care Qual Assur. 2007;20(2–3):184–194. [PubMed]
11. Plsek Paul E. Working Paper: some emerging principles for managing in complex adaptive systems. Paul E. Plsek and Associates, Inc. Uploaded to Internet 1997 November 5. Available from:
12. Lindberg C, Nash S, Lindberg C. On the Edge: Nursing in the Age of Complexity. Bordentown, NJ: Plexus Press; 2008.
13. Anderson RA, Crabtree BF, Steele DJ, McDaniel RR., Jr Case study research: the view from complexity science. Qual Health Res. 2005 May;15(5):669–685. [PMC free article] [PubMed]
14. Mahon J, Laupacis A, Donner A, Wood T. Randomised study of n of 1 trials versus standard practice. BMJ. 1996;312(7038):1069–1074. [PMC free article] [PubMed]
15. McDaniel RR, Jr, Jordan ME, Fleeman BF. Surprise, surprise, surprise! A complexity science view of the unexpected. Health Care Manage Rev. 2003;28(3):266–278. [PubMed]
16. Boustani M, Munger S, Beck R, Campbell N, Weiner M. A geroinformatics tool to enhance the care of hospitalized older adults with cognitive impairment. Clin Interv Aging. 2007;2(2):247–253. [PMC free article] [PubMed]
17. Boustani M, et al. Developing a CDSS to care for hospitalized elders with CI: the e-CHAMP Study. Alzheimer’s disease: New advances. 2007:169–172.
18. Cohen D, McDaniel RR, Jr, Crabtree BF, et al. A practice change model for quality improvement in primary care practice J Healthc Manag 2004. 493155–168.168; discussion 169–170.170 [PubMed]
19. Crabtree BF. Primary care practices are full of surprises! Health Care Manage Rev 2003. 283279–83.83; discussion 289–290.290 [PubMed]
20. Leykum LK, et al. Organizational interventions employing principles of complexity science have improved outcomes for patients with Type II diabetes. Implement Sci. 2007;2:28. [PMC free article] [PubMed]
21. Litaker D, Tomolo A, Liberatore V, Stange KC, Aron D. Using complexity theory to build interventions that improve health care delivery in primary care. J Gen Intern Med. 2006;21(Suppl 2):S30–S34. [PMC free article] [PubMed]
22. Miller WL, Crabtree BF, McDaniel R, Stange KC. Understanding change in primary care practice using complexity theory. J Fam Pract. 1998;46(5):369–376. [PubMed]
23. Miller WL, McDaniel RR, Jr, Crabtree BF, Stange KC. Practice jazz: understanding variation in family practices using complexity science. J Fam Pract. 2001;50(10):872–878. [PubMed]
24. Stroebel CK, et al. How complexity science can inform a reflective process for improvement in primary care practices. Jt Comm J Qual Patient Saf. 2005;31(8):438–446. [PubMed]
25. Miller WL, Crabtree BF. Healing landscapes: patients, relationships and creating optimal healing places. J Altern Complement Med. 2005;11(Suppl 1):S41–S49. [PubMed]
26. Solberg LI, Hroscikoski MC, Sperl-Hillen JM, Harper PG, Crabtree BF. Transforming medical care: case study of an exemplary, small medical group. Ann Fam Med. 2006;4(2):109–116. [PubMed]
27. Westfall JM, Mold J, Fagnan L. Practice-based research – “Blue Highways” on the NIH Roadmap. JAMA. 2007;297(4):403–406. [PubMed]
28. Boustani M, Healey P, Sennour Y, Munger S. Indianapolis Discovery Network for Dementia. J Am Geriatr Soc. 2007;55:S44.
29. Muir Gray JA, Haynes RB, Sackett DL, Cook DJ, Guyatt GH. Transferring evidence from research into practice: 3. Developing evidence-based clinical policy. ACP J Club. 1997;126(2):A14–A16. [PubMed]
30. NIH Roadmap for Clinical Research: Clinical Research Networks and NECTAR. Available from: Accessed April 17, 2007.
31. Zerhouni E. Medicine The NIH Roadmap. Science. 2003 Oct 3;302(5642):63–72. [PubMed]
32. Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ. 1998;317(7167):1185–1190. [PMC free article] [PubMed]
33. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. [PubMed]
34. Crabtree BF, et al. Delivery of clinical preventive services in family medicine offices. Ann Fam Med. 2005;3(5):430–435. [PubMed]
35. Indianapolis Discovery Network for Dementia [homepage on the Internet]. Available from: Accessed March 10, 2010.
36. Dans AL, Dans LF, Guyatt GH, Richardson S. Users’ guides to the medical literature XIV. How to decide on the applicability of clinical trial results to your patient. Evidence-Based Medicine Working Group. JAMA. 1998;279(7):545–549. [PubMed]
37. Guyatt GH, et al. Users’ guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA. 1995;274:1800–1804. [PubMed]
38. West S, et al. Rockville, MD: Agency for Healthcare Research and Quality; 2002. Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment No. 47 (Prepared by the Research Triangle Institute – University of North Carolina Evidence-based Practice Center under Contract No. 290-97-0011). AHRQ Publication No. 02-E016.
39. Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt GH. Users’ guides to the medical literature: VIII. How to use clinical practice guidelines. B. what are the recommendations and will they help you in caring for your patients? JAMA. 1995;274(20):1630–1632. [PubMed]
40. Faison WE, et al. Potential ethnic modifiers in the assessment and treatment of Alzheimer’s disease: challenges for the future. Int Psychogeriatr. 2007;19(3):539–558. [PubMed]
41. Guyatt G, et al. Determining optimal therapy – randomized trials in individual patients. N Engl J Med. 1986;314(4):889–892. [PubMed]
42. Guyatt GH, et al. The n-of-1 randomized controlled trial: clinical usefulness. Our three-year experience. Ann Int Med. 1990;112(4):293–299. [PubMed]
43. Larson EB, Ellsworth AJ, Oas J. Randomized clinical trials in single patients during a 2-year period. JAMA. 1993;270(22):2708–2712. [PubMed]
44. Callahan CM, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA. 2006;295(18):2148–2157. [PubMed]
45. Unutzer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002;288(22):2836–2845. [PubMed]
46. Counsell SR, Callahan CM, Buttar AB, Clark DO, Frank KI. Geriatric Resources for Assessment and Care of Elders (GRACE): a new model of primary care for low-income seniors. J Am Geriatr Soc. 2006;54(7):1136–1141. [PubMed]

Articles from Clinical Interventions in Aging are provided here courtesy of Dove Press