|Home | About | Journals | Submit | Contact Us | Français|
US colorectal cancer screening guidelines for people at average risk for colorectal cancer endorse multiple screening options and recommend that screening decisions reflect individual patient preferences.
We used the Analytic Hierarchy Process (AHP) to ascertain decision priorities of people at average risk for colorectal cancer attending primary care practices in Rochester NY, Birmingham AL, and Indianapolis IN. The analysis included four decision criteria, three sub-criteria, and ten options.
484 people completed the study; 66% were female, 49% were African-American, 9% had low literacy skills, and 27% had low numeracy skills. Overall, preventing cancer was given the highest priority (mean priority 55%), followed by avoiding screening test side effects (mean priority 17%), minimizing false positive test results (mean priority 15%), and the combined priority of screening frequency, test preparation, and the test procedure(s) (mean priority 14%). Hierarchical cluster analysis revealed six distinct priority groupings containing multiple instances of decision priorities that differed from the average value by a factor of four or more. More than 90% of the study participants fully understood the concepts involved, 79% met AHP analysis quality standards, and 88% were willing to use similar methods to help make important healthcare decisions.
These results highlight the need to facilitate incorporation of patient preferences into colorectal cancer screening decisions. The large number of study participants able and willing to perform the complex AHP analysis used for this study suggests that the AHP is a useful tool for identifying the patient-specific priorities needed to ensure that screening decisions appropriately reflect individual patient preferences.
Colorectal cancer is both a leading cause of morbidity and mortality and one of the most preventable cancers. [1, 2] Population-wide screening can reduce the adverse effects of colorectal cancer and is recommended in the United States and many other countries. [3–7]
Unfortunately, the difficulties implementing any widespread cancer screening program are compounded by the lack of a dominant colorectal cancer screening regimen.  Consequently, US screening guidelines endorse multiple options with different combinations of advantages and disadvantages and recommend that screening decisions take individual patient preferences and circumstances into account. [3–5, 9] A recent NIH consensus panel identified gaining a better understanding of patient preferences regarding colorectal cancer screening as one of the highest priorities for research in colorectal cancer prevention. 
Preference-based decisions are not uncommon and occur in many areas of human endeavor. Multi-criteria decision making methods are designed to help people make good decisions under these circumstances by helping them better understand the available information, assess their decision preferences and priorities, and enhance communication among involved stakeholders. [10, 11] These characteristics suggest that multi-criteria methods are useful tools for assessing individual patient and provider preferences and exploring how they affect clinical decisions. The results of several small pilot studies support this hypothesis. [12–16]
The goals of this study were to determine the feasibility of using the Analytic Hierarchy Process (AHP), a widely used multi-criteria method, to ascertain the colorectal cancer screening priorities of a large, diverse group of people at average risk for colorectal cancer and to explore associations between an individual’s decision priorities and their demographic characteristics, literacy, and numeracy skills.
People were eligible to participate if they were 49 to 85 years old and at average risk for colorectal cancer, regardless of their colorectal cancer screening history or current status.  People were excluded if they did not understand English well enough to complete the study interview, were unable to see the study materials, were cognitively impaired, had an acute illness, reported a medical condition that made them higher than average risk for colorectal cancer, or did not wish to participate.
We recruited study participants from primary care practices affiliated with Unity Health System in Rochester NY, Indiana University, Indianapolis IN, the University of Alabama at Birmingham, and Birmingham AL area practices participating in the Alabama Practice-Based Continuing Medical Education Network. Recruitment methods included signs, posters, physician recommendations, and asking people in office waiting rooms if they were interested. Before participating, everyone completed an informed consent document approved by the local institutional review board.
The study intervention was a four-part structured interview consisting of: a) an overview of current colorectal cancer screening recommendations; b) a multi-criteria decision analysis using the Analytic Hierarchy Process (AHP); c) collection of information about the participant including demographics, knowledge of colorectal cancer screening, literacy and numeracy; and d) an evaluation of the AHP-based priority assessment procedure. Interviews were conducted over a two year period from the spring of 2006 to the spring of 2008 by on-site research assistants and typically took about 45 minutes to complete.
We used the 2004 version of the “Facts on Screening” document published by the CDC’s National Colorectal Action Campaign to describe colorectal cancer, the rationale for screening, the recommended screening tests, the differences among them, and the lack of a dominant screening strategy. 
Based on published reports, the AHP is the most widely used multi-criteria method in the world.  It is a five-step process:
Because we were interested in assessing colorectal cancer screening priorities, our study focused on the first three of these steps. Operationalization of the AHP for this study is described below. A full description of the AHP is beyond the scope of this report; detailed descriptions of the method and its validation have been published previously. [19–23]
The goal of the AHP analysis presented to the study participants was to choose the best colorectal cancer screening strategy. Based on considerations included in US guideline statements at the beginning of the study (2006), we defined four major decision criteria: Prevent Cancer, Avoid Side Effects, Minimize False Positives, and Logistics. Logistics was further divided into three sub-criteria: Screening Frequency, Preparation for Screening, and the Screening Procedure. [24, 25] We did not include costs as a criterion because it was not technically feasible to generate individualized cost information for the entire range of study participants.
We included ten screening options: the six recommended options at the time of the study - annual guaiac-based fecal occult blood tests, annual immunochemical fecal occult blood tests, flexible sigmoidoscopy every five years, combined annual fecal occult blood tests and flexible sigmoidoscopy every five years, double-contrast barium enema every five years, and colonoscopy every ten years – and two additional tests that seemed likely to be included in future recommendations, CT colonoscopy and fecal DNA tests. (These two tests were added to updated screening recommendations published by the multidisciplinary guidelines panel in 2008. ) Because the reported test characteristics for guaiac fecal occult blood tests varied widely, we included two separate guaiac options with test sensitivities of 20% and 40% respectively. We also included combined fecal occult blood test and flexible sigmoidoscopy strategies using both guaiac-based fecal occult blood tests (with 40% sensitivity) and immunochemical-based fecal occult blood tests.
Participants used the standard AHP pairwise comparison method to compare the screening options and judge the relative priorities of the decision criteria. The option comparisons were done first so that participants would be familiar with the relative strengths and weaknesses of the options before they assessed the criteria priorities. This “bottom-up” approach is commonly used to compensate for situations where a simplifying assumption used in the AHP, that the relative priorities of the criteria are independent of the range of options being compared, does not hold. 
To guide the comparisons with respect to preventing cancer, avoiding side effects, and minimizing false positive test results, we provided participants with age-adjusted screening outcome estimates derived using an updated version of the simulation program developed for the 1997 Multi-Society guidelines. [26, 27] These estimates assumed screening with the same program at the prescribed interval from current age through age 80 and are included in an online appendix. To reduce the number of comparisons needed to compare the ten options relative to these three criteria, we divided each set of comparisons into three linked subsets - two with four options and one with three - that we subsequently combined to recreate the full comparison set.  This procedure reduced the number of required comparisons for each of these criteria from 45 to 15.
To ensure that the study participants compared the screening options using only the expected outcome data provided, the options were identified by an arbitrary letter during the analysis. The only exception was judgments regarding screening procedure, which had to be described for the comparisons to be made.
Overall, the analysis required 85 pairwise comparisons. All comparisons were made using an interactive computer program created for the study using Microsoft Excel and Crystal Xcelsius software that was validated using Expert Choice, a commercially available AHP software program. [29–31] An example is shown in Figure 1.
We used the standard AHP eigenvector method to calculate the priorities participants assigned to the decision criteria and sub-criteria. All AHP-derived scores are normalized so they sum to one and are expressed on a ratio scale.
The AHP also includes a measure of the internal consistency of a user’s comparison judgments which is used to judge the quality of the analysis. This measure, called the consistency ratio, compares the internal consistency of an observed set of judgments with that of a random set. A perfectly consistent matrix has a consistency ratio of 0. We defined a technically adequate analysis as a consistency ratio ≤ 0.15, the standard suggested for analyses performed by general population samples. [21, 32, 33] Current practice is to identify and correct substantially inconsistent judgments if possible. If decision makers are unable or unwilling to change their initial judgments enough to meet consistency standards, guidelines recommend that the decision be postponed until they develop a deeper understanding of the situation and their own preferences and priorities. 
We obtained participant demographic information using self-report. Study age was defined as the nearest five year age interval between 50 and 80 years. We measured literacy using the Rapid Estimation of Adult Literacy in Medicine (REALM) , numeracy using the Subjective Numeracy Scale, [36, 37] and knowledge of colorectal cancer prevention using a ten-item true-false test, a copy of which is included in an online supplemental file.
Study participants assessed how hard it was for them to understand and perform the AHP analysis using a five point scale ranging from 1 (not at all hard) to 5 (very hard). For analysis we combined responses 1 and 2 into a “not hard” category and responses 3, 4 and 5 into a “hard” category. They also used a five point scale ranging from 1 (not at all willing) to 5 (very willing) to indicate their willingness to use an AHP-based procedure to help them make better decisions about important health care issues. For analysis we combined responses 1, 2, and 3 into a “not willing” category and responses 4 and 5 into a “willing” category.
Because the priorities of both the decision criteria and sub-criteria are inter-related, we used hierarchical cluster analysis to describe how participants generated combined priorities for both sets of criteria. The clustering was based on the increase in sum of squares of the squared Euclidean distance of the untransformed priority scores. Because the priorities in the clusters are not normally distributed, we determined the statistical significance of differences in priorities among clusters using Kruskal-Wallis tests.
We assessed associations between participant characteristics, major criteria priorities, and consistency using nominal logistic regression. To achieve adequate sample size, we combined the cluster priorities into two groups, based on whether preventing cancer was the most important criterion or not. Independent variables included age, gender, race, education level, marital status, socio-economic status based on median household zip code income, literacy, numeracy, pre-study knowledge of colorectal cancer, and study site. We defined a statistically significant association as a p value ≤ 0.05.
Table 1 contains descriptive information about the 484 study participants. Three hundred seventy nine (78%) participants achieved technically adequate AHP analyses with consistency ratios ≤ 0.15. Except where noted, all of the results reported below are based solely on this group of participants.
The criteria priorities are summarized in Figure 2; additional details are provided in a supplemental online file. Overall, Prevent Cancer was the most important criterion with a mean priority score of 54%, followed by Avoid Side Effects (18%), Minimize False Positives (15%), and Logistics (12%). Of the logistical sub-criteria, Screening Procedure was given the highest mean priority, 44%, followed by Screening Frequency (32%), and Preparation for Screening (24%).
The cluster analyses identified six clusters of major decision criteria priorities and four clusters of logistical sub-criteria priorities. The priority differences among the clusters in both sets of criteria were statistically significant, with p < 0.001 for all comparisons. These results are illustrated in Figures 3 and 4. Additional details are provided in an online supplemental file.
The major criteria clusters can be combined into two groups: one consisting of patients who identified preventing cancer as the most important consideration but differed in the relative priorities of the other criteria (clusters 1–3) and one consisting of patients who identified a different criterion as most important (clusters 4–6). The majority of study participants, 341 (90%), were in the former group.
The logistical sub-criteria clusters similarly show a range of preferences. The largest cluster, containing 46% of the study population, consists of participants who judged the Screening Procedure as the most important of these considerations. However, clusters also exist where participants judged Screening Frequency and Preparation for Screening as the most important logistical sub-criterion as well as one where all three sub-criteria were judged equally important.
There were no significant associations between major criteria cluster group and participant age, gender, race, education level, marital status, literacy, numeracy, pre-study knowledge of colorectal cancer, or study site. Participants with median household incomes < $35,000 were more likely to be in a cluster where preventing cancer was not the most important criterion, 12% vs. 6%, p = 0.003.
There were no significant associations between a technically adequate analysis and participant age, race, education level, marital status, literacy, numeracy, pre-study knowledge of colorectal cancer, or median household income. Females were slightly less likely to achieve an adequate analysis than males: 77% vs 81% respectively, p = 0.008. The rates of adequate analyses also varied by study site: 32% for Indianapolis, 72% for Rochester, and 99% for Birmingham, p < 0.001.
Table 2 summarizes the results of the study participants’ evaluations of the AHP priority assessment method. These results include all study participants including those who did not achieve a technically adequate analysis. The proportions of all study participants who indicated that it was not hard to understand the decision criteria ranged from 92% to 93%. Ninety-one percent indicated it was not hard to understand the pairwise comparison process and 85% indicated it was not hard to make the comparisons. Finally, 88% indicated a willingness to use a similar procedure to help make an important healthcare decision. Compared with participants who were unable to achieve a technically adequate analyses, those who did indicated less difficulty making all comparisons and more willingness to use a procedure like this to make healthcare decisions. All differences between these two groups are statistically significant, p < 0.001.
A thorough understanding of individual patient preferences is important for both patient-centered care and informing future screening recommendations. Previous studies examining patient colorectal cancer screening preferences have reported a single most important attribute or presented summary information about attribute priorities obtained from an entire study sample. [40–44] They have also demonstrated that clinicians have difficulty predicting patient decision priorities. [40, 43] Our results extend these findings by demonstrating that patient decision priorities vary widely and cannot be predicted using demographic factors, numeracy, or literacy skills.
These results illustrate the importance of identifying individual preferences regarding the tradeoffs involved in choosing among currently recommended colorectal cancer screening options. Clinically, this information is essential for patient-centered decision-making. Collectively, this additional level of detail can be used to help develop screening policies and guidelines that are consistent with the priorities of the target population.
Our findings also provide new evidence that a clinical decision tool based on the AHP is feasible and has the potential to foster patient-centered decision making regarding colorectal cancer screening. Even though we used a comprehensive decision model that was challenging to analyze, 79% of the study participants met our standard for technical acceptability, more than 90% reported no difficulty understanding the conceptual basis of the analysis, and 88% indicated a willingness to use similar methods to help make important healthcare decisions. These findings provide important confirmatory evidence that similar results seen in smaller, single center studies are not unique but represent findings that can be extended to additional patients and practice settings. [13–15, 45]
Our study has several limitations. Because of the general recruitment methods used, we cannot ascertain how representative our sample is. This shortcoming, however, does not negate the implications of the wide variety of opinions expressed by study participants. Since we do not envision patients being forced to use decision aids, the impact of this limitation on the practical implications of our findings is likely to be small.
Another limitation is that we were unable include cost as one of the decision criteria. However, the absence of cost as a criterion does not negate the major findings of this study: that individuals vary in how they make tradeoffs among factors that differentiate among currently recommended colorectal cancer screening options in ways that are difficult to discern indirectly and that the AHP is a promising method for eliciting patients’ perspectives regarding these tradeoffs and integrating them into clinical decisions. Moreover, cost could easily be added as a criterion included in a revised model intended for additional research or implementation.
We also did not conduct a separate validation of the AHP-derived priorities beyond measuring judgmental consistency. This is, however, considered standard practice as the AHP has already been extensively validated. 
Finally, we cannot fully explain the effects of study site on the proportion of technically adequate analyses. All participants completed their analyses using the same computer program and our data indicate that patient age, education level, literacy level, and numeracy skill did not play a major role. These differences therefore highlight the importance of the local context in both studying and implementing a decision support intervention. They also suggest that regular provision of feedback and discussion regarding inconsistent judgments and allowing sufficient time for patients to assess their preferences and priorities are important features to include in multi-criteria based clinical decision support systems.
In summary, our findings emphasize the importance of patient-centered decisions regarding colorectal cancer screening and highlight the need to develop clinically feasible decision support methods to facilitate this process. The finding that many people are able and willing to perform a complex multi-criteria analysis using the Analytic Hierarchy Process suggests that AHP-based clinical decision support tools are feasible and appropriate for clinical use in assessing patient decision priorities. The proven effectiveness of the AHP in supporting individual and group decision making suggests that AHP-based clinical decision support tools have the potential to foster high-quality patient care decisions regarding not only colorectal cancer screening but any decision that depends on the successful integration of objective data, subjective judgments, and personal preferences and values. [46–48] A high priority for research is learning how to configure these tools so that they can effectively engage both patients and clinicians and be integrated into the clinical workflow. Ongoing work to restructure the practice environment such as the patient centered medical home initiative provides an opportunity to test new ways to implement innovative tools for making shared decision making a routine part of medical care. 
This study was supported by grant 1R01CA112366-1A1from the National Cancer Institute.
An abstract based on this study was presented at the Annual Meeting of the Society for Medical Decision Making in October, 2011.
Conflict of Interest
The authors have no conflicts of interest related to this study.