Case 1: Follow-up of an Abnormal Mammogram
This case involves a failure to ensure follow-up of abnormal screening tests within a health center that has multiple practices. A multilevel intervention to address this problem could include three levels: 1) an organizational level that addresses the medical and administrative leadership of an organization using an academic detailing model, 2) a team level to engage members of the health-care team in adopting skills in patient-centered communication and the appropriate use of the tracking system, and 3) a patient level that includes culturally appropriate materials and instructions regarding the meaning of the test results and how patients would have their abnormal screening test evaluated. The organizational level intervention would provide the leadership with information about the screening deficits at their facility and try to elicit their support to implement a tracking system to monitor the status of individuals with abnormal screening tests.
A number of the designs could be used to evaluate the intervention at various stages of development. In a preliminary study, fractional factorial designs could be used to evaluate components at any of the three levels. Multiple components could be tested, with data collected on intermediate outcomes, such as knowledge, attitudes, intentions, and perceived barriers. Components that appeared promising would be retained for the next level of testing.
As a next step, a TSD could be employed in a single health center to provide information on whether the multilevel intervention as a whole was associated with changes in follow-up test completion. The TSD would require that records be available frequently enough to make use of the analytic methods associated with the TSD.
Alternatively, a multiple baseline design involving a few health centers could be used even if data were not available on the frequency required for the time series. Data would have to be collected periodically to establish a stable baseline, and the centers would be given the intervention sequentially and in a random order. If the pattern in the outcome was linked to the intervention, and no similar change occurred absent the intervention, the investigators would have evidence for an intervention effect.
A GRT would provide the strongest evidence but would require multiple health centers randomized to either an intervention or a control arm. The size and cost of the study could be limited by sampling practices within health centers and patients within practices and delivering the team- and patient-level interventions only to those sampled. Even if it were necessary to deliver the interventions more broadly, sampling could be used to limit the scope and cost of data collection.
An additive design could be used if there were interest in the incremental effects of the three interventions. A four-arm design might be used with a usual care control; a patient-level arm; a patient- and team-level arm; and a patient-, team-, and organization-level arm. This would be the only design of the set that would provide information on the incremental effects of the three intervention levels.
Case 2: Implementing Electronic Medical Records
The head of a large health-care organization decides that she wants to implement the electronic medical record in a way that will allow her to evaluate its impact on organizational morale, provider team functioning, and patient care. The administrative leader decides to implement it in stages among the 50 facilities within her health-care organization. She recognizes it will be a disruptive process, so she hopes to measure care and the effects of implementation in 25 clinics for a year after a 6-month implementation period.
The facilities vary in size from 5 to 15 providers serving populations from 10
000 to 30
000 patients. The populations in these clinics make an average of 40
000 visits per year to the smallest clinics and 120
000 visits per year to the largest clinics. There are five clinics with five providers, 15 with seven providers, 10 with 10 providers, and 20 with 15 providers. On average, each provider team includes a receptionist, a licensed practical nurse, and a physician. There is one nurse for every five providers. This nurse has some responsibility for quality improvement activities.
The administrator decides to emphasize staff satisfaction, teamwork, and patient outcomes she increasingly values—breast, cervical, and colorectal cancer screening rates, diabetes management, and hypertension management. Staff satisfaction and teamwork would be measured by survey. Patient cancer-related outcomes would be measured using screening rates within 2 years among patients who have been seen in the clinic at least twice in 3 years and at least once in the past 1 year. The economic levels of the populations served by these clinics differ, and some are in rural settings.
This case involves a single intervention that is expected to have effects at several levels. It could be examined using a few health centers with a multiple baseline design, but as noted above, causal inference would rely on logic rather than between-center comparisons. Alternatively, this case is a natural setting for a GRT. Clinics might be stratified based on the number of providers and the socioeconomic status of the clinic population. If possible, they could also be stratified on urban vs rural. After baseline data collection for 6 months, clinics would be randomized to intervention or control, with 25 in each arm. Data collection would continue for another 6 months during the intervention. Trends in screening and other outcomes would be compared pre- and postintervention between the intervention and control arms. This would be a very strong design and likely have very good power.