Search tips
Search criteria 


Logo of hsresearchLink to Publisher's site
Health Serv Res. 2000 March; 34(7): 1519–1534.
PMCID: PMC1975658

A controlled time-series trial of clinical reminders: using computerized firm systems to make quality improvement research a routine part of mainstream practice.


OBJECTIVE: To explore the feasibility of conducting unobtrusive interventional research in community practice settings by integrating firm-system techniques with time-series analysis of relational-repository data. STUDY SETTING: A satellite teaching clinic divided into two similar, but geographically separated, primary care group practices called firms. One firm was selected by chance to receive the study intervention. Forty-two providers and 2,655 patients participated. STUDY DESIGN: A nonrandomized controlled trial of computer-generated preventive reminders. Net effects were determined by quantitatively combining population-level data from parallel experimental and control interrupted time series extending over two-month baseline and intervention periods. DATA COLLECTION: Mean rates at which mammography, colorectal cancer screening, and cholesterol testing were performed on patients due to receive each maneuver at clinic visits were the trial's outcome measures. PRINCIPAL FINDINGS: Mammography performance increased on the experimental firm by 154 percent (0.24 versus 0.61, p = .03). No effect on fecal occult blood testing was observed. Cholesterol ordering decreased on both the experimental (0.18 versus 0.1 1, p = .02) and control firms (0.13 versus 0.07, p = .03) coincident with national guidelines retreating from recommending screening for young adults. A traditional uncontrolled interrupted time-series design would have incorrectly attributed the experimental-firm decrease to the introduction of reminders. The combined analysis properly indicated that no net prompting effect had occurred, as the difference between firms in cholesterol testing remained stochastically stable over time (0.05 versus 0.04, p = .75). A logistic-regression analysis applied to individual-level data produced equivalent findings. The trial incurred no supplementary data collection costs. CONCLUSIONS: The apparent validity and practicability of our reminder implementation study should encourage others to develop computerized firm systems capable of conducting controlled time-series trials.

Full text

Full text is available as a scanned copy of the original print version. Get a printable copy (PDF file) of the complete article (1.7M), or click on a page image below to browse page by page. Links to PubMed are also available for Selected References.

Images in this article

Click on the image to see a larger version.

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Barr DA. The effects of organizational structure on primary care outcomes under managed care. Ann Intern Med. 1995 Mar 1;122(5):353–359. [PubMed]
  • Dietrich AJ, Carney PA, Winchell CW, Sox CH, Reed SC. An office systems approach to cancer prevention in primary care. Cancer Pract. 1997 Nov-Dec;5(6):375–381. [PubMed]
  • Donner A, Klar N. Methods for comparing event rates in intervention studies when the unit of allocation is a cluster. Am J Epidemiol. 1994 Aug 1;140(3):279–301. [PubMed]
  • Frame PS, Berg AO, Woolf S. U.S. Preventive Services Task Force: highlights of the 1996 report. Am Fam Physician. 1997 Feb 1;55(2):567–582. [PubMed]
  • Goldberg HI, Tarczy-Hornoch P, Stephens K, Larson EB, LoGerfo JP. Internet access to patients' records. Lancet. 1998 Jun 13;351(9118):1811–1811. [PubMed]
  • Landefeld CS, Aucott J. Improving primary care in academic medical centers. The role of firm systems. Med Care. 1995 Mar;33(3):311–314. [PubMed]
  • O'Connor GT, Plume SK, Olmstead EM, Morton JR, Maloney CT, Nugent WC, Hernandez F, Jr, Clough R, Leavitt BJ, Coffin LH, et al. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. The Northern New England Cardiovascular Disease Study Group. JAMA. 1996 Mar 20;275(11):841–846. [PubMed]
  • Ornstein SM. Electronic medical records in family practice: the time is now. J Fam Pract. 1997 Jan;44(1):45–48. [PubMed]
  • Rodgers GB. The safety effects of child-resistant packaging for oral prescription drugs. Two decades of experience. JAMA. 1996 Jun 5;275(21):1661–1665. [PubMed]
  • Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996 Nov-Dec;3(6):399–409. [PMC free article] [PubMed]
  • Tomford JW, Hershey CO, McLaren CE, Porter DK, Cohen DI. Intravenous therapy team and peripheral venous catheter-associated complications. A prospective controlled study. Arch Intern Med. 1984 Jun;144(6):1191–1194. [PubMed]

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust