PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of fampractLink to Publisher's site
 
Fam Pract. Dec 2011; 28(6): 683–688.
Published online May 12, 2011. doi:  10.1093/fampra/cmr026
PMCID: PMC3215921
Validity of self-reported colorectal cancer test use in different racial/ethnic groups
Navkiran K Shokar,a* Sally W Vernon,b and Carol A Carlsonc
aDepartment of Family and Community Medicine, Texas Tech University Health Sciences Center, 9849 Kenworthy, El Paso TX 79912
bUniversity of Texas-Houston School of Public Health, Division of Health Promotion and Behavioral Sciences, Houston, TX
cDepartment of Family Medicine, University of Texas Medical Branch, 301 University Boulevard, Galveston, TX 77555 1123, USA
*Correspondence to Navkiran K Shokar, Department of Family and Community Medicine, Texas Tech University Health Sciences Center, 9849 Kenworthy, El Paso TX 79912, USA; E-mail: navkiran.shokar/at/ttuhsc.edu
Received December 21, 2010; Revised April 1, 2011; Accepted April 7, 2011.
Objective. Prevalence of colorectal cancer (CRC) screening is ascertained by self-reported screening, yet little is known about the accuracy of this method across different racial/ethnic groups, particularly Hispanics. The purpose of this study was to compare the accuracy of CRC self-report measures across three racial/ethnic groups.
Methods. During 2004 and 2005, 271 white, African-American and Hispanic participants were recruited from a primary care clinic in Southeast Texas, and their CRC testing history based on self-report and medical record (the ‘gold standard’) were compared.
Results. Over-reporting was prevalent. Overall, up-to-date CRC test use was 57.6% by self-report and 43.9% by medical record. Racial/ethnic group differences were most pronounced for Hispanics in whom sensitivity was significantly lower for any up-to-date testing, fecal occult blood testing, flexible sigmoidoscopy and double contrast barium enema. There were no statistically significant differences across groups for over-reporting, specificity or concordance.
Conclusions. Self-report prevalence data are overestimating CRC test use in all groups; current measures are less sensitive in Hispanics.
Keywords: Colorectal carcinoma, cancer screening, primary care, African Americans, Hispanic Americans, reliability and validity
Population-based estimates of the prevalence of colorectal cancer (CRC) screening in the USA have largely relied on patient self-reports.1,2 Yet, accurate determination of CRC screening is particularly challenging because an array of tests are recommended at differing frequencies, recommendations continue to change over time,3,4 and patients are unfamiliar with the tests.5 Establishing the accuracy of self-report measures is therefore very important, particularly in different racial/ethnic groups, because differences in CRC outcomes may be attributable to differences in screening uptake. Although there have been a number of recent studies on the accuracy of self-reported CRC screening behaviours,610 only one compared the accuracy of self-report CRC measures in different ethnic groups;9 none has examined accuracy of self-report based on recent guidelines,4,11 in Hispanics. Given that disparities in CRC screening exist,1,2 it is particularly important to ensure that prevalence estimates are accurately measured in all these groups. It is also important to test the accuracy of these measures in different groups because self-report measures are often used to evaluate the effect of CRC screening studies and programs. The purpose of this study, therefore, is to compare the accuracy of self-reported CRC testing across three major racial/ethnic groups in a diverse clinic sample.
Sample and procedure
Participants in this study were part of a parent study identifying factors associated with CRC screening.12 Parent study participants were recruited from a university-based family medicine clinic in Southeast Texas over a 16-month period during 2004 and 2005. The clinic serves a diverse mix of racial/ethnic groups from both urban and semi-rural areas. To be eligible, patients had to be 50–80 years of age and of non-Hispanic white, African-American or Hispanic race/ethnicity. Individuals with a past history of CRC or high risk of CRC (familial adenomatous polyposis syndrome, hereditary non-polyposis CRC or ulcerative colitis) were excluded but those with other common gastrointestinal diagnoses (polyps in the bowel, irritable bowel syndrome, diverticulosis or abdominal hernia) were included. A quota sampling scheme, balanced by race/ethnicity, age (<65, ≥65) and gender, was instituted to increase the statistical power for comparisons by race/ethnicity and age. Participants were eligible for inclusion in the validation arm of the study if they had been a patient at the institution since the age of 50 or for the last 10 years (whichever period was shorter) and had medical records available. Participants for the validation study (n = 300) were randomly selected from this group (n = 560) The sample size was calculated by assuming sensitivity estimates of 89–96% and specificity estimates of 86–97%13 and a prevalence of screening of 40%, in which case, group sizes of 100 would provide sufficient power (alpha = 0.05 and beta = 0.20) to detect differences as small as 20%.
Data collection
Self-reported CRC screening was assessed using validated core measures with minor adaptation following pilot testing.14 These measures include a description to help patients identify each of the following tests: fecal occult blood testing (FOBT), colonoscopy (COL), flexible sigmoidoscopy (FS) and double contrast barium enema (DCBE) followed by a series of questions to establish which tests were performed and when they were done. A Spanish language version was developed using standard back translation methods.15 We also collected sociodemographic information including the patient’s age, educational level, gender, income and insurance type.16 Race/ethnicity was self-reported and elicited with a two-part question consistent with federal criteria.17 In addition, health status and previous gastrointestinal diagnoses were also collected.
Documented screening was determined by a comprehensive review of patients’ complete medical records including: (i) the hospital-wide paper chart that is used by all specialists and includes copies of all laboratory (FOBT), radiology (DCBE) and procedure (COL, FS) reports; (ii) the outpatient electronic medical record that captures all family medicine outpatient visits and (iii) the electronic lab database that was instituted in 2002 and captures all FOBT results, DCBE reports and pathology specimens from COL or FS. For each subject, a target date range for abstraction was established as being the date, they turned 50 to the time of interview or if >60 years, then the previous 10 years. The abstraction protocol was finalized after pilot testing and abstractors were trained by the principal investigator (PI) (N.K.S.) and research associate (C.A.C.). Five abstractors were used, the PI and research associate rechecked charts that were inconclusive to confirm final results by consensus. Although it is possible that patients could have had testing elsewhere, it is unlikely because of the unique geography of the area (an island with no availability of other secondary or tertiary care providers); patients tend to stay within the associated university system for their testing.
Analysis
Up-to-date screening in the medical record and by self-report was determined by whether the data from self-report or chart review revealed testing, for any reason according to the guidelines current at the time of the study:11 home FOBT within a year of the survey report or FS within 5 years or DCBE in the previous 5 years or COL in the previous 10 years. We evaluated sensitivity, specificity, raw agreement (concordance) and the report-to-records ratio of the self-report measures. These were calculated by determining the true positive, false positive, true negative and false negative rates by comparing the patient’s self-reported screening with the medical record. If a patient had multiple up-to-date tests of the same type, we used the most recent test.
‘Sensitivity’ was defined as the number of participants who had documentation of a test in the medical record who reported having had the test. ‘Specificity’ was defined as the proportion of those without a test in the record who reported not having the test. ‘Concordance’ was defined as agreement between self-report on the survey and the medical record. The ‘report-to-records ratio’ was defined as the total number of self-reported tests versus actual number documented. We calculated two-sided 95% confidence intervals around the estimates for sensitivity, specificity, concordance and report-to-records using standard methods.18,19 Performance measures were reported for any up-to-date test according to the recommendations and by test type for the overall sample and by racial/ethnic subgroup. We used Tisnado’s criteria20 to evaluate the sensitivity and specificity of outpatient services: where an estimate of <0.7 indicates poor agreement, ≥0.7 to <0.8 indicates fair agreement, ≥0.8 to <0.9 suggests good agreement and ≥0.9 indicates excellent agreement. Statistical significance was inferred by non-overlapping confidence intervals across tests and subgroups.
The response rate for the parent survey was 53%, with no significant differences between respondents and non-respondents for age, gender or race/ethnicity.12 From this sample, we identified 390 eligible cases and randomly chose 300 for inclusion in the validation study. During the abstraction, we found that 29 patients did not have records for the entire period and were therefore excluded from further analyses, leaving 271 patients in the study. Of these, 33% were African-American, 34% were Hispanic and 33% were white. Fifteen surveys were completed in Spanish. Racial/ethnic group differences were noted for education, income and health status (Table 1). In total, 57.6% (n = 156) of the patients reported themselves as being up-to-date with screening with at least one type of test, (whites: 65.2%, African-Americans 55.6% and Hispanics 52.2%), whereas 43.9% (n = 119) were up-to-date according to the medical record (whites: 49.4%, African-Americans: 44.4% and Hispanics: 38.0% (Table 1). Twenty per cent of subjects (n = 54/271) were up-to-date with more than one type of test.
TABLE 1
TABLE 1
Participant characteristics by race/ethnicity
For any up-to-date testing in the sample as a whole, sensitivity was good (0.85), specificity was poor (0.69), concordance was fair (0.79) and over-reporting was prevalent (1.25). Subgroup analyses revealed that sensitivity for any up-to-date testing was significantly lower in Hispanics compared to whites: sensitivity was only fair in Hispanics (0.77), good in African-Americans (0.83) and excellent in whites (0.93).
Examination by test type included the most recent up-to-date test of each type (so if there were multiple tests of same type, only the most recent was included, Table 2). In the entire sample, sensitivity varied by test type: the highest being good for COL (0.89), fair for FOBT (0.70) and poor for DCBE (0.45) and FS (0.5). The sensitivity for COL was significantly higher than for all three other tests. Specificity and concordance were overall in the good range (≥0.80) and were comparable across tests. Over-reporting was prevalent for all tests, but the only statistically significant difference was that over-reporting for DCBE was higher than for COL. Racial/ethnic subgroup analyses revealed that sensitivity in Hispanics was in the poor range for FOBT, FS and DCBE and was significantly lower than for African-Americans or whites for these tests; no African-American/white differences were noted. We found no differences in sensitivity for any of the tests based on the level of education of the participants. Specificity and concordance did not vary much across groups for any test and it was predominantly in the good range. All groups over-reported all tests but there were no statistically significant group differences observed.
TABLE 2
TABLE 2
Validity estimates for CRC screening according to recommendations
This study is one of the first to examine self-reported CRC testing accuracy for four recommended CRC screening tests in Hispanics compared to other groups. Our study suggests that there are significant differences in self-report measure sensitivity across racial/ethnic groups, but that all groups over-report tests. This finding is in keeping with prior qualitative work in this triethnic population that revealed patients had difficulty distinguishing between different CRC tests and often mentioned upper gastrointestinal tests as CRC tests as well.5 These survey items were designed to address that difficulty but may need further development to address this problem in Hispanics. We found only one study comparing accuracy of self-reported CRC test use in Hispanics with other groups21 and that study was conducted before the widespread dissemination and promotion of the guidelines that began in 199722 and did not examine COL or DCBE. They found no difference in sensitivity or specificity for either FOBT or FS measures. However, they examined only a 2-year time frame, and few FS tests were done in Hispanics. In a recent meta-analysis,23 self-reported accuracy of other types of cancer screening tests were compared across racial/ethnic groups. Mammography and digital rectal exam self-reports were less sensitive and Pap test self-reports were less specific in Hispanics compared to other groups. In our study, we observed better sensitivity for self-report of FOBT, FS and DCBE in African-Americans compared to Hispanics, but we did not observe statistically significant differences in any self-reported CRC test measure between African-Americans and whites. One other recent study comparing CRC self-report accuracy in African-American and white predominantly male veterans also found no differences.9
Our sample estimates of sensitivity and specificity for each of the four tests are in line with other recent studies examining the accuracy of CRC self-report.710 Consistent with our findings, they observed the highest sensitivity for COL recall (0.77–0.92), lowest for BE (0.49–0.74) with intermediate values for FS (0.75–0.87) and FOBT (0.56–0.93). These studies also found that the specificity of the tests varied much less across tests (range 0.72–0.97), and that there was over-reporting for all tests, particularly DCBE.
Strengths of our study include that it is one of the first to examine the accuracy of self-report CRC test use in Hispanics. We maximized the ascertainment of completed tests, through use of multiple sources of chart data, restricting our inclusion criteria to patients in the system for a sufficient length of time to adhere to the guidelines for all of the CRC tests and taking advantage of the relative geographic isolation of the university system that maximized the chances that patients did not go elsewhere for health care. However, there are some limitations of the study. Our response rate was modest at 53%, however, we did determine that our respondents were no different to non-respondents and were representative of the clinic population. Second, because we included only those with access to health care who had medical record availability, our findings should be generalized with caution to other populations and settings. Our estimates, however, were similar to those reported in other studies. Third, we were not able to analyze the characteristics of the Spanish version of the measures because of a small sample size. Fourth, because of our small sample size, we did not control for racial/ethnic group differences in education, income or health status that may have influenced our findings. Although a separate analysis by educational level across all groups showed no differences in sensitivity measures.
Developing surveys items for multiethnic groups is fraught with difficulties24,25 because responding to a survey is a complex process. It has been conceptualized as involving four different stages, each of which can lead to inaccuracies in reporting. During the ‘comprehension’ stage, the respondent interprets the meaning of the question; in the ‘retrieval’ stage, the respondent relies on long-term memory for relevant information; in the ‘estimation/judgment’ stage, the respondent assesses the information retrieved and its relevance to the question and chooses to accept or reject the information. In the final ‘response stage’, the respondent weighs factors such as sensitivity and social desirability of the responses and then decides what answer to give. Cultural differences among groups could affect the process at any of these stages and may have done so in our population.26 Previous studies have reported ethnic group differences in response style24 and have reported that measures of preventive behaviors are particularly problematic across cultures.25 These measures were cognitively tested but not in Hispanics in the target age group. Future studies should cognitively test measures in all groups and ages.27
The implication of these findings is that national prevalence estimates maybe overstating screening rates in all groups, and that the observed racial/ethnic group differences in test use are real, especially for FOBT, FS and DCBE. Overall the accuracy of these self-report measures is acceptable: with COL self-report displaying the least racial/ethnic variation in accuracy. Since secular trends in CRC test use point to greater use of COL and declining rates of FS FOBT1,28 and DCBE29 in the USA, the prevalence of inaccurate group estimates may be attenuated as COL becomes the preponderant CRC test. However, further challenges in accurately measuring CRC screening test use will remain, such as the addition of new tests to the recommendations.3 Self-report measures of new tests will need to be developed and our work points to the vital importance of cognitive testing of new measures in all population subgroups to enhance understanding of the cultural influences on the cognitive, emotional and judgment processes utilized by survey respondents.
Declaration
Funding: John Sealy Memorial Endowment Fund for Biomedical Research; and the National Cancer Institute (grant no. K07 CA107052).
Ethical approval: The study was IRB approved by the University of Texas Medical Branch Institutional Review Board.
Conflict of interest: none.
Acknowledgments
We would like to thank Alma Salazar, Mary Jane Simmons and Gustavo Andrade for assistance with data collection.
1. Joseph D, Rim S, Seeff L. Use of colorectal cancer tests—United States, 2002, 2004, 2006. MMWR. 2008;299:2501–2.
2. Shapiro JA, Seeff LC, Thompson TD, et al. Colorectal cancer test use from the 2005 National Health Interview Survey. Cancer Epidemiol Biomarkers Prev. 2008;17:1623–30. [PubMed]
3. Levin B, Lieberman DA, McFarland B, et al. Screening and surveillance for the early detection of colorectal cancer and adenomatous polyps, 2008: a joint guideline from the American Cancer Society, the US Multi-Society Task Force on Colorectal Cancer, and the American College of Radiology. CA Cancer J Clin. 2008;58:130–60. [PubMed]
4. Whitlock EP, Lin JS, Liles E, Beil TL, Fu R. Screening for colorectal cancer: a targeted, updated systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2008;149:638–58. [PubMed]
5. Shokar N, Weller S, Vernon S. Cancer and colorectal cancer: knowledge, beliefs and screening preferences of a diverse population. Fam Med. 2005;31:341–7. [PubMed]
6. Bastani R, Glenn BA, Maxwell AE, et al. Validation of self-reported colorectal cancer (CRC) screening in a study of ethnically diverse first-degree relatives of CRC cases. Cancer Epidemiol Biomarkers Prev. 2008;17:791–8. [PubMed]
7. Jones R, Mongin S, Lazovich D, Church T, Yeazel M. Validity of four self-reported colorectal cancer screening modalities in a general population: differences over time and b. Cancer Epidemiol Biomarkers Prev. 2008;17:777–84. [PubMed]
8. Khoja S, McGregor SE, Hilsden RJ. Validation of self-reported history of colorectal cancer screening. Can Fam Physician. 2007;53:1192–7. [PMC free article] [PubMed]
9. Partin M, Grill J, Noorbaloochi S, et al. Validation of self-reported colorectal cancer screening behavior from a mixed-mode survey of veterans. Cancer Epidemiol Biomarkers Prev. 2008;17:768–76. [PubMed]
10. Vernon S, Tiro JA, Vojvodic RW, et al. Reliability and validity of a questionnaire to measure colorectal cancer screening behaviors: does mode of survey administration matter? Cancer Epidemiol Biomarkers Prev. 2008;17:758–67. [PubMed]
11. Pignone M, Rich M, Teutsch S, Berg A, Lohr K. Screening for colorectal cancer in adults at average risk: a summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2002;137:132–41. [PubMed]
12. Shokar NK, Carlson CA, Weller SC. Factors associated with racial/ethnic differences in colorectal cancer screening. J Am Board Fam Med. 2008;21:414–26. [PubMed]
13. Baier M, Calonge N, Cutter G, et al. Validity of self-reported colorectal cancer screening behavior. Cancer Epidemiol Biomarkers Prev. 2000;9:229–32. [PubMed]
14. Vernon S, Meissner H, Klabunde C, et al. Measures for Ascertaining Use of Colorectal Cancer Screening in Behavioral, Health Services, and Epidemiologic Research. Cancer Epidemiol Biomarkers Prev. 2004;13:898–905. [PubMed]
15. Marin G, Marin B. Research with Hispanic Populations. Newbury Park, CA: Sage; 1991.
16. Prevention CfDCa. Behavioral Risk Factor Surveillance System Survey Questionnaire. US Department of Health and Human Services, Centers for Disease Control and Prevention; 2000. http://www.cdc.gov/BRFSS (accessed on 15 June 2001)
17. Anonymous. Office of Management and Budget: Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity. 1997. web: OMB Home http://www.whitehouse.gov/omb/fedreg/ombdir15html.
18. Fleiss J. Statistical Methods for Rates and Proportions. 2nd edn. New York, NY: John Wiley; 1981.
19. Harper R, Reeves B. Reporting of precision of estimates for diagnostic accuracy: a review. BMJ. 1999;318:1322–3. [PMC free article] [PubMed]
20. Tisnado DM, Adams JL, Liu H, et al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44:132–40. [PubMed]
21. Hiatt RA, Pérez-Stable EJ, Quesenberry C, et al. Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California. Prev Med. 1995;24:278–85. [PubMed]
22. Byers T, Levin B, Rothenberger D, Dodd G, Smith R. American Cancer Society guidelines for screening and surveillance for early detection of colorectal polyps and cancer: update 1997. American Cancer Society Detection and Treatment Advisory Group on Colorectal Cancer. CA Cancer J Clin. 1997;47:154–60. [PubMed]
23. Rauscher G, Johnson TP, Cho YI, Walk JA. Accuracy of self-reported cancer-screening histories: a meta-analysis. Cancer Epidemiol Biomarkers Prev. 2008;17:748–57. [PubMed]
24. Warnecke RB, Johnson TP, Chávez N, et al. Improving question wording in surveys of culturally diverse populations. Ann Epidemiol. 1997;7:334–42. [PubMed]
25. Pasick RJ, Stewart SL, Bird JA, D'Onofrio CN. Quality of data in multiethnic health surveys. Public Health Rep. 2001;116:223–43. [PMC free article] [PubMed]
26. Willis G, editor. Cognitive Interviewing and Questionnaire Design. American Association for Public Opinion Research—57th Annual Conference, 2002. NCI-NIH: St. Pete Beach, FL;
27. Burgess D, Powell AA, Griffin JM, Partin MR. Race and the validity of self-reported cancer screening behaviors: development of a conceptual model. Prev Med. 2009;48:99–107. [PubMed]
28. Meissner H, Breen N, Klabunde C, Vernon S. Patterns of colorectal cancer screening uptake among men and women in the United States. Cancer Epidemiol Biomarkers Prev. 2006;15:389–94. [PubMed]
29. Zhao B, Kilbourne B, Stain SC, et al. Racial disparities and trends in use of colorectal procedures among Tennessee elderly (1996–2000) Ethn Dis. 2006;16:412–20. [PubMed]
Articles from Family Practice are provided here courtesy of
Oxford University Press