Search tips
Search criteria 


Logo of procamiaLink to Publisher's site
AMIA Annu Symp Proc. 2009; 2009: 558–562.
Published online 2009 November 14.
PMCID: PMC2815413

Provider Perceptions of Colorectal Cancer Screening Clinical Decision Support at Three Benchmark Institutions

Jason J. Saleem, PhD,1–3 Laura G. Militello, MA,4 Nicole Arbuckle, BA,4 Mindy Flanagan, PhD,2 David A. Haggstrom, MD, MAS,1,2,7 Jeffrey A. Linder, MD, MPH,5,6 and Bradley N. Doebbeling, MD, MSc1,2,7


Implementation of computerized clinical decision support (CDS), and its integration into workflow has not reached its potential. To better understand the use of CDS for colorectal cancer (CRC) screening at benchmark institutions for health information technology (HIT), we conducted direct observation, including opportunistic interviews of primary care providers, as well as key informant interviews and focus groups, to document current challenges to CRC screening and follow-up at clinics affiliated with the Veterans Heath Administration, Regenstrief Institute, and Partners HealthCare System. Analysis revealed six common barriers across institutions from the primary care providers’ perspective: receiving and documenting “outside” exam results, inaccuracy of the CDS, compliance issues, poor usability, lack of coordination between primary care and gastroenterology, and the need to attend to more urgent patient issues. Strategies should be developed to enhance current HIT to address these challenges and better support primary care providers and staff.


Computerized clinical decision support (CDS), through the use of an electronic health record (EHR), can improve clinician decision-making, support adherence with evidence-based guidelines, and ultimately improve quality of care1,2. However, the integration of CDS into clinical workflow has not reached its potential3. One approach for understanding this missed opportunity is examining barriers to effectively using CDS from the providers’ perspective. In this study, the CDS was computerized or paper clinical reminders, or an electronic template, coordinated with an EHR. We chose to focus our study of barriers to effective use of CDS for colorectal cancer (CRC) screening and follow-up.

CRC ranks third among causes of cancer deaths, and is the third most common cancer among both men and women in the US. CRC has a significant economic impact on health care systems, patients, families, and society. The total costs attributed to CRC in the US is approximately $6 billion, with 80% of these due to inpatient medical care costs, making CRC among the costliest cancers to treat4,5. Stage at diagnosis is the primary predictor of survival. Unfortunately, less than half (40%) of colorectal cancers are found at an early stage, in large part due to low rates of screening; in 2004, approximately 57% of those eligible were screened for CRC6. There is strong evidence that colorectal cancer screening can reduce mortality from colorectal cancer7. With low screening rates and evidence for screening effectiveness, the CRC screening process is an ideal opportunity for improving the integration of CDS into outpatient clinical workflow. A report by RAND Corporation estimates that “properly” implemented HIT could prevent 17,000–38,000 deaths from CRC each year.8

A recent systematic review on CDS identified the Veterans Heath Administration (VHA), Regenstrief Institute (RI), Partners HealthCare System (PHS), and Intermountain Healthcare as the four benchmark institutions most frequently cited with high quality research demonstrating the efficacy of CDS in improving quality and efficiency9. Because CDS is so widely implemented in these institutions, they provide an ideal health care setting in which to study provider perceptions of CDS that may influence CRC screening and follow-up. To better understand the use of CDS for CRC at benchmark institutions, we conducted a qualitative field study to document current challenges to CRC screening and follow-up.


Site Selection

We selected three of the four benchmark institutions for health IT for the present study: VHA, RI, and PHS9. Two Veterans Affairs Medical Center (VAMC) sites were selected based on having a strong medical informatics research presence, strong clinical performance, and being geographically distributed nationally (south and east). At each site, qualitative data was collected in multiple outpatient clinics. For both RI and PHS, outpatient clinics were not in the same building so observations occurred at multiple community outpatient clinics.

Field Study Methods

The researchers conducted direct observation (with opportunistic interviews) of CDS use, as well as key informant interviews and focus groups, at two VAMC sites, RI, and PHS to identify putative best practices and barriers to effective of CRC CDS for the various modalities of CRC screening: fecal occult blood test (FOBT), flexible sigmoidoscopy, and colonoscopy. Table 1 summarizes the participants across each site and number of provider-patient encounters observed.

Table 1.
Number of participants across study sites. VAMC = Veterans Affairs Medical Center; RI = Regenstrief Institute; PHS = Partners HealthCare System; phys = physician; res = resident; NP = nurse practitioner; PA = physician assistant.

Direct Observation

The researchers used direct observation to understand the range of ways in which providers interact and use CDS tools in real time. Direct observation of CDS use and integration into workflow in real time allowed researchers to gather data on the context and process surrounding CDS use. During observations, two to four observers experienced in ethnographic observation separately shadowed providers as they interacted with CDS tools during an actual work shift. Observations were recorded via handwritten notes on a structured observation form during participant interaction with the CDS, capturing observable activities and verbalizations.

Observers also conducted opportunistic interviews of providers on their use of CDS in the outpatient clinics to better understand the observational data. These interviews were conducted so as not to disrupt the natural workflow of the providers. This discussion covered why providers took certain actions as well as opinions and feedback about barriers to the use of CRC CDS. This opportunistic feedback was recorded in the structured observational form. This feedback supplemented and aided understanding of corresponding observations.

Key Informant Interviews and Focus Groups

The content of key informant interviews covered mechanisms and best practices used to facilitate CDS implementation and integration into workflow. While the same core questions were asked for each interviewee, the semi-structured nature allowed for flexibility and gave the interviewee an opportunity to elaborate on, or cover important topics that would not have otherwise surfaced. Sample questions include the following: At what point do you interact with clinical reminders for outpatients? What is your ideal workflow in the outpatient clinic? What difficulties have you experienced fitting use of clinical reminders into your optimal workflow? Key informants were identified as clinical champions for CDS and/or CRC screening. In addition, focus groups were conducted at the two VHA sites.

Data Collection

Before each site visit, a local contact person was identified who served as the liaison during the visit. This person introduced the observers and scheduled the observations in outpatient clinics. For each site, investigators conducted observations during two full days in at least two different outpatient clinics. Providers included in the observations read and signed an informed consent if they chose to participate in the study. The handwritten observations were typed after each site visit, and a scheme applied to permit tracking of observer, site, and day.

At each site, key informants were identified from the outpatient clinics. The key informant interviews were conducted either in-person during the site visit or afterward by phone. At the two VHA sites, all providers who participated in the observations were invited to participate in the focus groups. Focus groups of 5–6 providers were facilitated by one of the observers to explore in depth barriers to CRC screening and follow-up. The study was approved by the Indiana University Institutional Review Board, the Indianapolis VA Medical Center Research Committee, and each individual study site.

Data Analysis

All data from the opportunistic interviews during observation, key informant interviews, and focus groups were analyzed using a coding template. The research team developed this coding template based on the sociotechnical model10 and a literature review. The coding template included a category for each component of the sociotechnical model: social subsystem, technical subsystem, external subsystem. For each of these categories, subcategory labels were identified. The coding template (or codebook) was modified as coding proceeded and themes emerged from the data. Findings were integrated across sites into meaningful patterns and the data abstracted into emergent themes (i.e., barriers to CRC screening and follow-up), as guided by qualitative analysis norms11.


The technical aspects of the design of the EHRs and CDS across the benchmark institutions is quite variable. Detailed descriptions of each institution’s EHR have been previously published1,12,13. The VHA sites deploy CRC CDS in the form of a computerized clinical reminder. One of the two VHA sites recently implemented a more complex set of CRC computerized clinical reminders which provides decision support not only for CRC screening, but also for follow-up and surveillance. The RI system used a paper encounter form reminder for CRC screening; these paper reminders were automatically generated by CDS rules in the EHR but presented to the provider in paper form. At the time of this data collection, PHS included CRC screening as part of an electronic, template health maintenance list, although a computerized clinical reminder for CRC screening is in development for near-future implementation. Despite varying forms of EHRs and CDS, our analysis revealed six common barriers to CRC screening and follow-up across these three benchmark institutions (Table 2).

Table 2.
Barriers to CRC screening and follow-up at the study sites: convergence across data collection methods. Three sources of data were analyzed: opportunistic interviews (OI) during provider observation, key informant interviews (KII), and focus group (FG) ...

Receiving and Documenting “Outside” Exam Results

Problems were reported with receiving and documenting results of colonoscopy exam performed at a different institution. At one site, a physician noted: “In the CRC clinical reminder [dialog] box you cannot easily document that a colonoscopy was done outside of the VA. Say the patient had an outside colonoscopy done 5 years ago- you need to enter the exact date, time, location. But the patient may only remember that he had a colonoscopy about 5 years ago.” Problems were also reported for colonoscopy exam reports from other institutions, such as the lack of specific recommended actions.

CRC CDS Not Accurate

Inaccuracy of the CDS for certain patients was reported as a barrier at sites that used patient-specific clinical reminders. A physician at one site reported: “One patient was sent to GI three times for a colonoscopy. Each time they told him he wasn’t due. But the reminder keeps coming up. He had a colonoscopy recently, so I don’t know why the reminder doesn’t turn off.”

Compliance Issues

Providers across study sites reported weak links in the CRC screening and follow-up process involving both clinic staff and patients. In one case, a staff member routinely did not distribute FOBT cards to patients at check-in, even though the computer system indicated that the cards were given to the patient. At a different site, another physician reported that for even the patients who agree to a colonoscopy, about half of those patients do not show up for their scheduled colonoscopy exam. “Patient memory” was also cited as a weak link in the process in terms of reliance on patient memory for date and results of last colonoscopy when that information is not detailed in the EHR.

Poor EHR or CDS Usability

Problems were reported with the usability of the EHR or CDS such as not having the appropriate options in a dialog box to satisfy the CRC reminder and an inability to easily track the date and results of last colonoscopies for patients in the EHR. One physician assistant (PA) relied on a self-made paper spreadsheet to track these results for all of his patients. Another case included a nurse practitioner (NP) not being able to see the CRC screening findings from the nursing intake exam (CRC screening is routinely started in the nursing intake exam in the VHA). The NP noted that if the health tech enters CRC screening information into the EHR after she has already opened the patient’s record, she can’t see the findings from intake. She either has to walk down to intake and ask, or repeat the screening. She generally chooses to repeat the screening as it seems more disruptive for her to walk down the hall.

Lack of Coordination between Primary Care and Gastroenterology (GI)

Coordination between the primary care provider and GI is a large part of the CRC screening and follow-up process. Potential coordination problems included distribution of responsibility and receiving exam results. For example, one primary care physician felt overburdened with being solely responsible for satisfying the CRC clinical reminders, noting: “GI should be able to clear out the [computerized clinical] reminder. For example, the patient we just saw…it took me a while to go through and satisfy it [the CRC clinical reminder]. The patients see lots of different people in the hospital and they all have their hands in the patient’s care. They should be satisfying some of the reminders as well.”

Delay in receiving colonoscopy results from GI was also an issue at one of the sites. A physician noted: “There is a four or five day delay between the [colonoscopy] result and when it shows up in [the EHR] because the GI docs have to have their note transcribed and then they approve it before it goes to [the EHR]. At another site, conversely, the GI physicians simply emailed the result directly to the primary care provider. One physician noted: “…if the colonoscopy is scheduled at 7:00am, the GI doc will often email the results right after so I’ll get them at 7:45am.” However, these results are not immediately documented in the EHR and available to other clinicians.

Acute vs. Preventive Care

Time pressure and competing demands were also factors in providing preventive services, including CRC screening. One physician noted that a discussion about CRC screening seems like a distraction when she is trying to help a patient with a very urgent problem. The physician explained: I have to choose between chest pain and hemoccult [FOBT], I am going to choose chest pain.” In contrast, at a clinic from a different site, an 85% CRC screening completion rate was achieved for eligible patients by adding a dedicated staff member to specifically track CRC screening and follow-up, as well as other preventive screening interventions.


Challenges to effective CRC screening and follow-up persist at three benchmark institutions for health IT. Each of our study sites relied on CDS (computerized or paper reminders, or an electronic template), coordinated with an EHR. Our results showed challenges to CRC screening and follow-up at all stages of the process, including screening, receiving laboratory results, and following up with the patient. We describe our findings in this context.

Challenges to Effective CRC Screening

The first point in the process, where the CRC screening recommendation is made, was problematic in several ways. The design of the EHR and CDS was a barrier to effective CRC screening recommendations when the CDS (i.e., the clinical reminders for CRC screening) were not accurate for the patient, and/or when the usability of the computer tools was poor (e.g., not having the appropriate options in the dialog box for the provider to accurately satisfy the clinical reminder). Also, CRC screening was not performed in some cases when providers faced high time pressure and/or when they perceived that the patient had more serious problems that needed attention (i.e., acute vs. preventive services). Staff-related compliance issues in the process were also identified at this point, such as distribution of FOBT cards to the patient.

Challenges to Effective Flow of CRC Laboratory Results to Primary Care Provider

Another major hurdle in the process from the primary care providers’ perspective was receiving results from the colonoscopy exam or other CRC screening tests. Challenges in receiving and documenting results from colonoscopy exams performed “outside” of the institution were reported at all study sites. Often, the patients were responsible for relaying this information to their primary care provider; however, reliance on patient memory was a barrier in this instance, especially in cases where the exam took place several years prior. Coordination with the commonly-used GI service, internal to the institution, was also reported as a challenge, such as delays in receiving results. Although not reported as a common barrier across sites, an EHR at one site that used a blend of paper charting posed a challenge to some providers in knowing where to find the most current CRC screening results, as some providers relied on paper charting and others relied on computerized charting.

Challenges to Effective Patient Follow-up

Finally, after receiving the results, the primary care providers reported challenges in following up with the patient in some cases. Again, “outside” exam results was a barrier. Challenges in receiving and documenting results from colonoscopy exams performed “outside” of the institution were reported at all study sites. However, when these results were documented and available to the primary care provider, there were sometimes problems with patient follow-up. For example, the report from the “outside” colonoscopy exam did not always include recommended actions.


We collected qualitative data at three benchmark institutions for HIT, VHA, RI, and PHS, to understand provider perspectives on challenges to effective CRC screening and follow-up, as related to the EHR and CDS systems. Although these institutions have varying forms of EHRs and CDS, our preliminary, ongoing analysis revealed common barriers, spanning CRC screening, receiving laboratory results, and following up with the patient. These challenges to effective CRC screening suggest that design enhancements can be made to the HIT, such as improved usability and integration into workflow, to reduce or eliminate some of these challenges and better support the primary care providers and clinic staff. Further study is needed to examine applicability of these results to institutions with less experience in CDS.


This research was supported by the Agency for Healthcare Research and Quality (AHRQ) contract HSA2902006000131. Dr. Saleem is supported by a VA HSR&D Career Development Award (CDA 09-024-1). The views expressed in this article are those of the authors and do not necessarily represent the view of the Department of Veterans Affairs.


1. Bates DW, Pappius E, Kuperman GJ, et al. Using information systems to measure and improve quality. Int J Med Inf. 1999;53:115–24. [PubMed]
2. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330:765. [PMC free article] [PubMed]
3. Garg AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA. 2005:1223–38. [PubMed]
4. Riley GF, Potosky AL, Lubitz JD, Kessler LG. Medicare payments from diagnosis to death for elderly cancer patients by stage at diagnosis. Med Care. 1995;33:828–41. [PubMed]
5. Taplin SH, Barlow W, Urban N, et al. Stage, age, comorbidity, and direct costs of colon, prostate, and breast cancer care. J Natl Cancer Inst. 1995;87:417–26. [PubMed]
6. Seeff LC, King J, Pollack LA, Williams KN. Increased Use of Colorectal Cancer Tests --- United States, 2002 and 2004. MMWR. 2006;55:308–11. [PubMed]
7. Bond JH, Mandel JS, Church TR. Role of fecal occult-blood testing. N Engl J Med. 1998;339:774–5. [PubMed]
8. Hillestad R, Bigelow JH. Health Information Technology: Can HIT Lower Costs and Improve Quality?RB-9136. 2005. RAND Corporation.
9. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–52. [PubMed]
10. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care--an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14:542–9. [PMC free article] [PubMed]
11. Roth EM, Patterson ES. Using Observational Study as a Tool for Discovery: Uncovering Cognitive Demands and Adaptive Strategies. In: Montgomery H, editor. How Professionals Make Decisions. Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc; 2005. pp. 379–93.
12. Brown SH, Lincoln MJ, Groen PJ, Kolodner RM. VistA--U.S. Department of Veterans Affairs national-scale HIS. Int J Med Inform. 2003;69:135–56. [PubMed]
13. McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inform. 1999;54:225–53. [PubMed]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association