|Home | About | Journals | Submit | Contact Us | Français|
To describe unexpected challenges and strategies to overcome them when conducting randomized controlled trials (RCT) of health services research interventions in retail pharmacies.
Thirty-six retail drug stores in Indianapolis.
We conducted an RCT to evaluate the effectiveness of an intervention to increase pharmacists’ involvement in caring for customers. We describe: (1) our RCT as originally designed, (2) unexpected challenges we faced; and (3) how we resolved those challenges.
Randomized controlled trial.
Major modifications in research design were necessitated by factors such as corporate restructuring, heightened sensitivity to patient confidentiality, and difficulties altering employees’ behavior. We overcame these barriers by conducting research that is consistent with corporate goals, involving appropriate corporate administrators and technical personnel early in the process, and being flexible.
Health services researchers should conduct RCTs in a variety of nonacademic practice settings to increase generalizability and better reflect the true impact of interventions. Pragmatic problems, although significant, can be successfully overcome.
Health services researchers who conduct randomized controlled trials (RCT) evaluate interventions to improve the organization, delivery, quality, and/or outcomes of care. Often, such trials are implemented in academic or other clinical settings in which the investigator has a reasonable degree of control over factors essential to conducting a well-designed RCT. Maximizing such control is important to enhancing the internal validity of an RCT. However, in so doing, findings may be less externally valid (i.e., generalizable) to nonacademic practice settings. This tension between internal and external validity is especially important to health services researchers.
We report our experience with conducting an RCT in retail pharmacies. After describing our RCT as originally designed, we present some unexpected situations that are unlikely to be experienced by health services researchers who conduct RCTs in more traditional venues. We then discuss how we resolved those challenges through sometimes-dramatic changes in the study protocol. Our experiences may be useful to other investigators who plan to evaluate health services interventions in nonacademic practice settings.
We evaluated the effectiveness of a pharmaceutical care program for patients with reactive airways disease (i.e., asthma or chronic obstructive pulmonary disease with a reversible component) who fill their prescriptions in retail drug stores. Such programs seek to increase pharmacists’ involvement in improving patients’ outcomes, rather than simply dispensing medications (Hepler and Strand 1990). Specifically, it was hypothesized that pharmacists can help optimize drug therapy by: educating patients; implementing strategies to foster medication compliance; coordinating drug therapy prescribed by multiple physicians; monitoring adverse drug events; being available to address patients’ questions and concerns; and ensuring that patients know when, who, and how to contact their physicians (Hepler and Strand 1990; Strand et al. 1991). Unfortunately, several recent literature reviews suggest that enthusiastic reports about the effectiveness of pharmaceutical care are often plagued by serious design flaws, particularly a lack of well-designed RCTs (Hatoum and Akhras 1993; Kennie, Schuster, and Einarson 1998; Singhal, Raisch, and Gupchup 1999; Tett, Higgins, and Armour 1993). Moreover, these reviews find no well-designed studies supporting the effectiveness of pharmaceutical care in retail pharmacies. We conducted the RCT in chain pharmacies because, as the largest provider of prescription services in the United States (Latner 2000), they provide a venue through which effective pharmaceutical care programs can have the greatest impact on patients’ lives. Moreover, effective programs that are successfully implemented in a few stores can efficiently be disseminated throughout a retail chain, thereby rapidly enhancing their impact upon patients’ lives. We targeted reactive airways disease because of its prevalence, morbidity, mortality, responsiveness to appropriate medication therapy, and associated health care costs (Ferguson and Cherniack 1993; Gourley, Gourley et al. 1998; Gourley, Portner et al. 1998; McFadden and Gilbert 1992; Murray, Stang, and Tierney 1997; Solomon et al. 1998; Stroupe, Gaskins, and Murray 1999; Weiss and Sullivan 1993).
The study was to be conducted in Revco pharmacies. With 99 stores and more than 200 pharmacists, Revco pharmacies filled half of the prescriptions for 1.5 million persons living in central Indiana. Importantly, Revco was participating in the Indianapolis Network for Patient Care (INPC). Funded by the National Library of Medicine National Information Infrastructure Initiative, INPC was designed as a model integrated network that linked data from Indianapolis’ major hospitals, neighborhood health centers, and public health and homeless clinics, as well as Revco drugstores (Overhage, McDonald, and Tierney 1995). For each medication filled, Revco transmitted prescription information to INPC each night (approximately twenty thousand prescriptions per night). We planned to capitalize on the INPC infrastructure both to identify eligible patients (by prescriptions filled) and collect data on emergency department and hospital visits made during the study period.
Patients would be eligible if they: (1) had reactive airways disease (i.e., filled a prescription for methylxanthines, inhaled corticosteroids, inhaled or oral sympathomimetics, inhaled parasympathetic antagonists, or inhaled cromolyn, and confirmed the diagnosis as an active problem during enrollment); (2) were ≥18 years of age; (3) received at least 70 percent of their medications from a single Revco store; (4) had no significant impairment in speech, vision, or hearing that prevented their being interviewed; (5) resided in the community; and (6) provided written informed consent. Initially, we intended to identify potential patients using Revco medication records stored in INPC. The remaining criteria would be assessed during telephone screening.
The core of the pharmaceutical care program involves pharmacists receiving clinically relevant, patient-specific data that allows them to use their skills and training to improve patient outcomes. Thirty-six Revco stores proximal to a major Indianapolis hospital contributing data to the INPC were divided into twelve clusters of three stores. The three stores within each cluster were matched on geographic location, percent of Medicaid-insured adults with reactive airways disease (a proxy for socioeconomic status), and the number of prescriptions filled (high versus low volume). Within each cluster, stores were randomly assigned to one of three study groups: (1) pharmaceutical care program group, (2) peak flow meter monitoring group, or (3) usual care control group.
Essential to our conception of pharmaceutical care was pharmacists’ having access to clinically relevant, patient-specific data. Thus, patients in the pharmaceutical care program group received a peak flow meter and personalized instruction about its use during the baseline interview. Then, during monthly telephone interviews conducted by research assistants, patients were asked to use the meter and report the peak flow result. The Revco-owned computer present in all stores to dispense medications was to alert pharmacists when a study patient filled any prescription. When this occurred, pharmacists were instructed to go to a separate study computer to review patient-specific information relevant to our pharmaceutical care program (i.e., peak flow results, recent emergency department visit, recent hospital admission, data on breathing medications).
Because persons in the pharmaceutical program group were called monthly to obtain a current peak flow result, we were concerned that these calls alone could constitute an intervention, that is, patients might increase their self-care due to greater monitoring of their disease activity. Thus, we included a peak flow meter only control group in which patients also received a peak flow meter and instructions about its use. However, to separate the effects of our program from increased monitoring, peak flow data in this group were not provided to the pharmacist. The usual care control group did not have peak flow rates assessed during monthly interviews.
After completing and pilot testing the program (including recruitment procedures), all pharmacists received training in April 1997. Enrollment was projected to begin in May 1997.
Literally within days of completing pharmacist training, CVS Pharmacy purchased Revco. The impact of this event was immediate and substantial:
Soon after acquiring Revco, CVS reviewed Revco's practices related to sharing data. This was clearly appropriate, given that academic and lay media brought national attention to many complex issues surrounding patient confidentiality. Thus, CVS halted data flow to INPC so that they could thoughtfully consider the implications of providing data to INPC. It is critical to note that Indiana law allows data transfer between two health care providers (e.g., CVS and INPC), with the stipulation that recipients of these data (i.e., INPC) are bound by the same rules of confidentiality as the health care provider transmitting the data. In the weeks that followed, CVS adopted a policy to safeguard patients’ confidentiality in a manner even more protective than that required by Indiana law. Specifically, CVS prohibited transfer of any prescription data to INPC (or released to us) without patients’ signed permission (“opt-in”). This made it impossible to use our original strategy to identify potential patients using prescription data from INPC. Over the next months, we worked closely with CVS to develop a recruitment strategy that, although more complex, balanced scientific and privacy concerns. The protocol, which was implemented in July 1998 after being approved by CVS and the Institutional Review Board at Indiana University-Purdue University at Indianapolis, was as follows (Figure 1):
From the outset, we thought it was essential that investigators be in close contact with stores in the pharmaceutical care group. Thus, three of the investigators were each assigned 3–5 intervention stores to visit periodically to identify any problems in the field and encourage full participation in the program. Early on, it became apparent that some pharmacists were not compliant with the various components of the program. Thus, we initiated strategies to enhance compliance. We began by monitoring the percentage of patients for whom pharmacists: (a) viewed data in the study computer and (b) took actions consistent with pharmaceutical care (e.g., made comments in the field available, distributed handouts). These data comparing pharmacists to their colleagues in other stores were faxed to pharmacists weekly, with the hope that this would encourage pharmacists to use data in the study computer. This, alone, was not successful. As a consequence, their direct supervisors were also provided these data and asked to follow-up with pharmacists who did not participate in the program. This strategy also was unsuccessful. Our final strategy was to offer pharmacists incentives ($50 gift certificates each month) if they met or exceeded predefined levels of compliance (viewing data for 90 percent of patients and documenting pharmaceutical care for 75 percent of patients). Pharmacists in the two stores with the highest compliance rates over the study period would receive an additional incentive of $100. These incentives appeared to encourage pharmacists to implement our program.
While it is not possible to determine the precise barriers to implementation, we have several recommendations that may be useful to others. First, the program needs to be as convenient as possible. In our particular case, it was not possible to integrate our program into the regular CVS store computers. Thus, we provided a separate study computer that contained the patient-specific data. Advances in web-based applications (including firewalls) may now permit integration of such programs into the store computer. Second, we trained all intervention pharmacists to deliver our program. Soon after training, it became evident that pharmacists were not uniformly interested in implementing the program. A better strategy may have been to identify one person in each store who was both enthusiastic about delivering pharmaceutical care and responsible for implementing it at his or her store. Third, implementation of the program must be viewed by pharmacists as part of their jobs. Thus, with acceptance of responsibility for the program, corporations may consider providing incentives for performance using measures reflecting either implementation of the program or actual patient outcomes.
We believe that RCTs of health services research interventions conducted in nonacademic practice settings can have a major impact on our health care system. While our experience comes from retail pharmacies, we believe that many lessons we learned may be applicable to other nonacademic practice settings. Based on our experience, we believe we can offer other investigators some advice.
In summary, conducting RCTs in a variety of practice settings should continue to be a goal for health services researchers. When hearing about our experiences, one might expect us to discourage investigators from pursuing this line of research. Nothing could be further from the truth. While we were delayed in starting our study, we completed recruitment in December 1999, having enrolled 1,113 patients. Moreover, our follow-up has been outstanding, as some 85 percent of patients have completed the 6-month follow-up. We had a similarly excellent rate for completing the 12-month closeout interviews. Moreover, we are already in the process of capitalizing on our established relationship with CVS to conduct additional studies related to pharmaceutical care for patients with heart failure.
The authors wish to acknowledge the contributions of Clement McDonald, Larry Lemmon, Robert Thompson, Birgit Erickson, Paul Traficante, Marina Weisburd, Faye Smith, Carol DeHaven, Rosalie DeMoss, Elizabeth Jobe, Michael Del Busto, David White, Randall Miller, Stacey Hart, Andy Arland, and Laura Garza.
This study was funded by a grant from the Agency for Healthcare Research in Quality (5 R01 HS09083). Dr. Weinberger was supported by the Health Services Research and Development Service, Department of Veterans Affairs, through a Research Career Scientist Award.