PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Arch Intern Med. Author manuscript; available in PMC 2012 April 11.
Published in final edited form as:
PMCID: PMC3169179
NIHMSID: NIHMS249952

Electronic Patient Messages to Promote Colorectal Cancer Screening: A Randomized, Controlled Trial

Abstract

Background

Colorectal cancer is a leading cause of cancer mortality, yet effective screening tests are often underused. Electronic patient messages and personalized risk assessments delivered via an electronic personal health record could increase screening rates.

Methods

We conducted a randomized, controlled trial in 14 ambulatory health centers involving 1,103 patients ages 50 to 75 with an active electronic personal health record who were overdue for colorectal cancer screening. Patients were randomly assigned to receive a single electronic message highlighting overdue screening status with a link to a web-based tool to assess their personal risk of colorectal cancer. The outcomes included colorectal cancer screening rates at 1 and 4 months.

Results

Screening rates were higher at 1 month for patients who received electronic messages compared to those who did not (8.3% vs. 0.2%, p<0.001), but this difference was no longer significant at 4 months (15.8% versus 13.1%, p=0.18). Among 552 patients randomized to receive the intervention, 296 (54%) viewed the message, and 47 (9%) used the web-based risk assessment tool. Among 296 intervention patients who viewed the electronic message, risk tool users were more likely than non-users to request screening examinations (17% versus 4%, p=0.04) and to be screened (30% versus 15%, p=0.06). One-fifth (19%) of patients utilizing the risk assessment tool were estimated to have above-average risk for colorectal cancer.

Conclusions

Electronic messages to patients produce an initial increase in colorectal cancer screening rates, but this effect is not sustained over time.

Keywords: Colorectal cancer screening, reminder systems, quality improvement, randomized controlled trial, electronic health record, personal health record

Colorectal cancer is the second leading cause of cancer mortality in the United States.1 Colorectal cancer screening detects cancers at more curable early stages and reduces colorectal cancer mortality.2-6 As a result, national guidelines strongly recommend screening with colonoscopy, flexible sigmoidoscopy, or fecal occult blood testing for average-risk adults over the age of 50.7, 8

Many patients do not receive recommended screening exams.9 The barriers to colorectal cancer screening are multi-faceted, with patients often citing lack of motivation and awareness of the need for colorectal cancer screening, as well as reporting that their provider did not recommend screening.10, 11 Direct-to-patient mailings can increase rates of colorectal cancer screening by heightening awareness and facilitating performance of recommended tests.12 However, such mailing programs cannot deliver information to patients in a highly personalized and interactive manner, and many patients remain unscreened even in the presence of such programs. One promising strategy could be to provide patients not only with information on overdue screening exams, but to highlight their personal risk of cancer, as those that perceive their own risk of colorectal cancer to be high are more likely to get screened.13

Innovative use of health information technology offers the potential to overcome the limitations of paper mailings and limited availability of personalized risk information. Personal health records directly integrated into electronic health records allow the efficient and direct delivery of information from doctors to patients.14 Such information can engage patients and encourage them to promote their own health behaviors. However, data are limited regarding how to use personal health records effectively in this manner.15 Therefore, we conducted a randomized, controlled trial to assess the impact of electronic patient messages coupled with personalized risk assessments delivered via an integrated personal health record on the rate of colorectal cancer screening.

Methods

Study Setting

This 4-month trial was conducted from November 2009 to March 2010 at Harvard Vanguard Medical Associates (HVMA), a multi-specialty group practice composed of 14 ambulatory health centers in eastern Massachusetts. Since 1997, clinical practices within HVMA have used a common electronic health record (Epic Systems, www.epicsystems.com) that includes clinical notes, diagnostic codes, procedure codes, and laboratory results. Screening tests for colorectal cancer can be ascertained via an automated electronic algorithm using laboratory results, diagnostic codes, procedure codes, and outpatient and hospital encounters from the electronic health record.

The electronic health record also supports a personal health record through an integrated web-based patient portal, referred to as “MyHealth”. This patient portal allows bidirectional electronic messaging between patients and their doctor's office. The portal also allows patients to view their current medication list and recent laboratory results and request medication refills and office appointments. Patients voluntarily choose to register for a password-protected personal health record account, which allows access to all of this information via the patient portal in the electronic health record.

Patient Eligibility

We identified 10,283 patients ages 50 to 75 years of 174 primary care physicians at 14 health centers who had a visit with an HVMA primary care physician during the prior 18 months and an active MyHealth personal health record account (Figure 1). Among these patients, 1,851 (18%) of 155 physicians were overdue for colorectal cancer screening based on the absence of a recorded colonoscopy within the prior 10 years, flexible sigmoidoscopy within the prior 5 years, or fecal occult blood testing within the prior 12 months. Because the electronic patient messages were designed to come from individual primary care physicians, we next provided each of the eligible physicians with a list of their eligible patients overdue for colorectal cancer screening who also had active MyHealth accounts. Each physician was asked to approve patients who should receive the electronic outreach message from the physician, with 80% of physicians responding. Our final study cohort included 1,103 patients of 109 physicians, representing 60% of the 1,851 potentially eligible patients.

Figure 1
CONSORT diagram of patient eligibility and randomization. Patients were randomized within health center and physician panels.

The Harvard Medical School and HVMA Human Studies Committees approved the study protocol, including a waiver of informed consent for patients because the study was promoting the HVMA standard of care for colorectal cancer screening. The study protocol was registered at ClinicalTrials.gov (ID number NCT01032746).

Patient Intervention

The intervention was randomized at the level of individual patients within each health center and each physician's patient panel. Patients overdue for colorectal cancer screening in the intervention group received a personalized electronic message from their primary care physician through the integrated personal health record messaging system. The message served several functions, including 1) highlighting their overdue screening status and providing information on screening options, 2) facilitating electronic requests to the clinical practice to arrange specific screening exams, and 3) providing a link to a specially tailored version of a validated web-based tool that enables patients to assess their risk of colorectal cancer (www.yourdiseaserisk.com) within their secure personal health record.

The two screening options recommended by local guidelines in the HVMA medical group included colonoscopy or home fecal occult blood testing with 3 stool cards. Patients could reply to the electronic message to request colonoscopy or to have a fecal occult blood test kit mailed to their home. For colonoscopy requests, a physician assistant screened patients by telephone for contraindications, provided instructions for bowel preparation, and scheduled the procedure. Gastroenterologists performed procedures either at an ambulatory endoscopy center operated by HVMA or at an affiliated hospital-based endoscopy center. The waiting time for colonoscopy at HVMA during the study intervention was approximately 3 weeks.

Patients randomized to the control group did not receive electronic messages. The organization does have a strong commitment to colorectal cancer screening, including the use of programs such as paper mailings to patients overdue for screening and the delivery of electronic reminders to physicians.12 However, these interventions were deactivated for over one year prior to the initiation of our current study. There were no additional programs or tools to promote colorectal cancer screening during the study period.

Patient Risk Website

The electronic message to patients contained a hyperlink to a modified version of a validated web-based tool that enabled patients to assess their personal risk of colorectal cancer (www.yourdiseaserisk.com).16 This modified web tool focused on colorectal cancer and was hosted on the internal server of HVMA to protect patient confidentiality and allow tracking of patient responses. Patients using the risk tool provided their age, sex, family history of colorectal cancer, relevant personal medical history, and diet and exercise patterns. This information was used to assess their personalized risk of colorectal cancer on a 7-point ordinal scale ranging from very-much-below average to very-much-above average relative to other individuals of the same age and sex. The risk status was displayed using a graphical format that demonstrated how much their risk could be lowered based on a normal colorectal cancer screening exam (Figure 2). Patients were given the option of sharing their risk information with their primary care physician on the final web-page of the risk tool.

Figure 2
Patients were provided with comparative risk assessment for colorectal cancer when using the web based risk tool. Adapted from www.yourdiseaserisk.com.

Prior to receiving their personalized risk assessment, patients were asked to respond to the statement “I intend to be screened for colon cancer within the next year” on a 4-point Likert scale from “Strongly Agree” to “Strongly Disagree”. Following receipt of their risk assessment, patients responded to the question “Would you recommend this Web site to your friends?” using a dichotomous yes/no response.

Study Outcomes

All data were collected from the electronic health record. The primary study outcome was receipt of appropriate colorectal cancer screening assessed at 4 months. Appropriate screening exams included colonoscopy, flexible sigmoidoscopy, or a 6-specimen home fecal occult blood test kit. We chose 4 months to allow for scheduling and performance of colonoscopy, as well as to ensure that any immediate effects on screening rates were sustained. We defined a secondary outcome of receipt of colorectal cancer screening at 1 month to assess any immediate effects of the electronic patient outreach.

We also collected information on uptake of the intervention components. The electronic messaging function allowed for tracking of whether patients viewed the message, and the web-based risk tool captured whether patients accessed the website and completed the risk assessment.

Data Analysis

Baseline characteristics of patients in the intervention and control groups were compared using the Pearson chi-square test for dichotomous variables and Student's t test for continuous variables. We analyzed the impact of the interventions by fitting a logistic regression model to predict performance of an appropriate screening exam. We adjusted standard errors for clustering of patients by health center using generalized estimating equations. The independent variable in this model was patient intervention status. All analyses were performed using SAS version 9.2, and we report two-tailed p values or 95% confidence intervals for all comparisons.

Results

Study Subjects

Patients in the intervention and control groups were similar according to age, sex, race/ethnicity, and insurance coverage (Table 1). The median number of eligible patients per physician was 9 (interquartile range 5 to 13). During the 4-month intervention period, the proportion of patients with at least one primary care office visit was the same in the intervention and control groups (46% versus 44%), and screening exams for colorectal cancer were ordered by primary care clinicians for 28% of patients.

Table 1
Baseline Patient Characteristics

Intervention Effect on Screening Rates

Among 552 patients randomized to receive the intervention, 255 (46%) viewed the electronic message within 1 month, 296 (54%) viewed the electronic message within 4 months, and 17 (3%) made electronic requests for colorectal cancer screening. At 1 month colorectal screening rates were significantly higher in the intervention group compared to the control group (8.3% versus 0.2%, p<0.001, Table 2). However, the primary endpoint at 4 months showed no difference between the intervention and control arms in rates of colorectal cancer screening (15.8% versus 13.1%, p=0.18). Among all patients, most (79%) new screening exams were colonoscopy, with fecal occult blood testing (21%) accounting for the remainder.

Table 2
Receipt of Colorectal Cancer Screening by Intervention Status

Web-Based Risk Tool

Among 296 intervention patients who viewed the electronic message, 47 (16%) accessed and completed the web-based tool. Among the 47 patients who used the web tool, three-quarters (74%) strongly or somewhat agreed they intended to be screened for colon cancer within the next year. Approximately one-fifth (19%) had above-average risk of colorectal cancer, with 11% having average risk and 70% demonstrating below average risk. No patients elected to have their risk information shared with their primary care physician, though 84% would recommend the web tool to friends.

Among the 296 intervention patients who viewed the electronic message, risk tool users were more likely than risk tool non-users to request screening examinations (17% versus 4%, p=0.04) and tended to be more likely to be screened (30% versus 15%, p=0.06).

Comment

In a cohort of average-risk adults, electronic outreach via an integrated personal health record produced an initial increase in colorectal cancer screening rates, but this effect was not sustained. Only half of patients in the intervention group viewed the electronic messages despite having it sent from their personal physician, which limited the overall effectiveness of the intervention. Users of the web-based risk assessment tool represented a small and generally low risk subset of all eligible patients, however they responded favorably to the tool and more often requested and received colorectal cancer screening exams than patients who were invited to use the tool but did not do so.

Increased screening is essential to reduce the incidence, morbidity, and mortality of colorectal cancer.17 Informed decision making and education can motivate patients to pursue appropriate screening.18-21 However, primary care physicians face increasing pressures to provide a wide range of preventive counseling during increasingly shorter office visits.22 Colorectal cancer screening presents a particular challenge given the multiple screening options, and many patients do not receive adequate counseling for colorectal cancer screening. 10, 23

Prior research has suggested that patient-directed interventions including videotaped decision aids,24 educational mailings,12, 25, 26 and nurse counseling27 can increase colorectal cancer screening rates. Even in the presence of such programs many patients remain unscreened, and such programs can have substantial costs.23, 28

Personal health records offer the promise of engaging patients in a more efficient and interactive manner.14, 29, 30 Patients have expressed interest in interacting with their medical record using electronic portals similar to the one used in our intervention.31 However, rigorous evaluations of targeted uses of such systems to improve care have been limited in number and had mixed results.15 One pilot study of an e-mail outreach program failed to increase colorectal cancer screening rates, but only 25% of patients viewed the messages.32 In another study within a large health maintenance organization, electronic messages delivered within an integrated personal health record found that two-thirds of patients viewed the message, and colorectal cancer screening rates were improved at 3 months.33

Our study builds on this prior work in important ways. First, about half (54%) of patients in our study viewed the personalized electronic messages, a rate that was intermediate relative to these two prior studies of personalized messages.32, 33 One of the significant challenges to the effective use of personal health records to improve quality of care is not only to increase their adoption by physician practices, but also to ensure effective use by patients.34 The solutions to this issue may be straightforward, such as ensuring accurate email addresses; or more complex, such as ensuring that the information and services provided by the personal health record meet the needs of patients. Our study highlights that while personal health records are often promoted to improve quality of care, it is important to evaluate such systems to ensure they are implemented in a manner that achieves this goal.

Second, we demonstrated a promising initial increase in screening, similar to a prior study,33 however we found that this effect was not sustained longer term. This may have been due to the effects of prior programs in our system to promote colorectal cancer screening, including annual paper mailings to adult patients overdue for colorectal cancer screening.12 Future work should focus on the impact of quarterly or semi-annual electronic outreach to patients on their screening rates, and how such regular outreach is received by patients. Third, the baseline screening rate in our population was very high (82%); exceeding that in a successful prior study of electronic patient outreach (69%),33 leaving an unscreened population of patients who may have been much more resistant to screening. As colorectal cancer screening rates rise across the country,9 it is increasingly important to understand how to promote screening in this remaining unscreened group.

Finally, we incorporated a link to a web-based risk tool to further personalize the information regarding colorectal cancer screening as many high-risk patients are not screened appropriately.35 Most patients report having the greatest trust in health information received from their physician, but many turn to the Internet for convenience, particularly for cancer-related information.36, 37 We employed a model that combined the trust of a recommended information source from the physician's office with the convenience of an Internet-based tool. We chose one validated web-based risk tool that was available when our study was implemented, and other options including the National Cancer Institute Colorectal Cancer Risk Assessment Tool can also be used to provide patients with risk information.38, 39 Those who accessed the tool in our study found it helpful and often sought screening afterward, but only a small proportion of patients used the tool. While providing accurate information in an easily interpretable format should remain essential goals of Internet-based tools,40, 41 future work also needs to focus on how to engage patients in using such tools when recommended by their physicians.

Our study findings should be considered in the context of some limitations. First, we relied on physicians to approve the inclusion of their patients in the study, and their approval was not obtained for 40% of eligible patients. This factor represents an important consideration for future work involving this type of direct patient outreach focused on quality improvement. Involvement of physicians prior to such patient outreach can provide an important safety screen, however it can also represent a barrier to quality improvement, particularly if physicians do not respond to requests to enroll their patients.

We do not have long-term follow up of patients regarding colorectal cancer outcomes, nor do we have information regarding patient perceptions of receiving electronic messages or of using the online risk assessment tool. The risk assessment tool we used provided level of risk relative to other age-matched peers, and therefore may have downplayed the steadily increasing risk of developing colorectal cancer after age 50.

Finally, some issues may limit the generalizability of our study findings to other settings and patient populations. The organization we studied has a history of focusing on colorectal cancer screening,12 and it has already achieved a relatively high rate of colorectal cancer screening. Also, patients actively using personal health records, such as those enrolled in our trial, may differ from the broader population according to race, ethnicity or socioeconomic status,42 as well as in their enthusiasm and comfort with using web-based applications. In the large medical group we studied, approximately 18% of patients ages 50 to 75 years old were active users of the personal health record in late 2009. Future research will be needed to understand how our results extend to other settings.

In conclusion, this randomized trial of electronic messages to patients within an integrated personal health record demonstrated an initial increase in the rate of colorectal cancer screening that was not sustained over time. However, only about half of eligible patients viewed the electronic message, and fewer than 10% accessed the web-based risk tool. Further research is needed to understand the most effective ways for patients to use interactive health information technology to improve their care and reduce the morbidity and mortality of colorectal cancer.

Acknowledgements

The authors are grateful to Amy Marston and Debby Collins for assisting with project management; and to the patients and physicians of Harvard Vanguard Medical Associates for their participation in this study. This study was supported by grant R01 CA112367 from the National Cancer Institute. The funding agency played no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. Dr. Sequist had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Supported by grant R01 CA112367 from the National Cancer Institute. This research was presented at the Annual Meeting of the Society of General Internal Medicine, Minneapolis, MN (April 30, 2010).

References

1. Jemal A, Siegel R, Ward E, Hao Y, Xu J, Thun MJ. Cancer statistics, 2009. CA Cancer J Clin. 2009;59:225–249. [PubMed]
2. Mandel JS, Bond JH, Church TR, et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study. N Engl J Med. 1993;328:1365–1371. [PubMed]
3. Hardcastle JD, Chamberlain JO, Robinson MH, et al. Randomised controlled trial of faecal-occult-blood screening for colorectal cancer. Lancet. 1996;348:1472–1477. [PubMed]
4. Kronborg O, Fenger C, Olsen J, Jorgensen OD, Sondergaard O. Randomised study of screening for colorectal cancer with faecal-occult-blood test. Lancet. 1996;348:1467–1471. [PubMed]
5. Selby JV, Friedman GD, Quesenberry CP, Jr., Weiss NS. A case-control study of screening sigmoidoscopy and mortality from colorectal cancer. N Engl J Med. 1992;326:653–657. [PubMed]
6. Winawer SJ, Zauber AG, Ho MN, et al. Prevention of colorectal cancer by colonoscopic polypectomy. The National Polyp Study Workgroup. N Engl J Med. 1993;329:1977–1981. [PubMed]
7. Levin B, Lieberman DA, McFarland B, et al. Screening and surveillance for the early detection of colorectal cancer and adenomatous polyps, 2008: a joint guideline from the American Cancer Society, the US Multi-Society Task Force on Colorectal Cancer, and the American College of Radiology. CA Cancer J Clin. 2008;58:130–160. [PubMed]
8. United States Preventive Service Task Force Screening for colorectal cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2008;149:627–637. [PubMed]
9. Centers for Disease Control Use of colorectal cancer tests--United States, 2002, 2004, and 2006. MMWR Morb Mortal Wkly Rep. 2008;57:253–258. [PubMed]
10. Klabunde CN, Vernon SW, Nadel MR, Breen N, Seeff LC, Brown ML. Barriers to colorectal cancer screening: a comparison of reports from primary care physicians and average-risk adults. Med Care. 2005;43:939–944. [PubMed]
11. Berkowitz Z, Hawkins NA, Peipins LA, White MC, Nadel MR. Beliefs, risk perceptions, and gaps in knowledge as barriers to colorectal cancer screening in older adults. J Am Geriatr Soc. 2008;56:307–314. [PubMed]
12. Sequist TD, Zaslavsky AM, Marshall R, Fletcher RH, Ayanian JZ. Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med. 2009;169:364–371. [PMC free article] [PubMed]
13. Palmer RC, Emmons KM, Fletcher RH, et al. Familial risk and colorectal cancer screening health beliefs and attitudes in an insured population. Prev Med. 2007;45:336–341. [PubMed]
14. Tang PC, Lee TH. Your doctor's office or the Internet? Two paths to personal health records. N Engl J Med. 2009;360:1276–1278. [PubMed]
15. Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs). J Am Med Inform Assoc. 2008;15:729–736. [PMC free article] [PubMed]
16. Kim DJ, Rockhill B, Colditz GA. Validation of the Harvard Cancer Risk Index: a prediction tool for individual cancer risk. J Clin Epidemiol. 2004;57:332–340. [PubMed]
17. Vogelaar I, van Ballegooijen M, Schrag D, et al. How much can current interventions reduce colorectal cancer mortality in the U.S.? Mortality projections for scenarios of risk-factor modification, screening, and treatment. Cancer. 2006;107:1624–1633. [PubMed]
18. Leard LE, Savides TJ, Ganiats TG. Patient preferences for colorectal cancer screening. J Fam Pract. 1997;45:211–218. [PubMed]
19. Pignone M, Bucholtz D, Harris R. Patient preferences for colon cancer screening. J Gen Intern Med. 1999;14:432–437. [PMC free article] [PubMed]
20. Ling BS, Trauth JM, Fine MJ, et al. Informed decision-making and colorectal cancer screening: is it occurring in primary care? Med Care. 2008;46:S23–29. [PubMed]
21. Manne SL, Coups EJ, Markowitz A, et al. A randomized trial of generic versus tailored interventions to increase colorectal cancer screening among intermediate risk siblings. Ann Behav Med. 2009;37:207–217. [PMC free article] [PubMed]
22. Yarnall KS, Pollak KI, Ostbye T, Krause KM, Michener JL. Primary care: is there enough time for prevention? Am J Public Health. 2003;93:635–641. [PubMed]
23. Holden DJ, Jonas DE, Porterfield DS, Reuland D, Harris R. Systematic review: enhancing the use and quality of colorectal cancer screening. Ann Intern Med. 2010;152:668–676. [PubMed]
24. Pignone M, Harris R, Kinsinger L. Videotape-based decision aid for colon cancer screening. A randomized, controlled trial. Ann Intern Med. 2000;133:761–769. [PubMed]
25. Myers RE, Sifri R, Hyslop T, et al. A randomized controlled trial of the impact of targeted and tailored interventions on colorectal cancer screening. Cancer. 2007;110:2083–2091. [PubMed]
26. Church TR, Yeazel MW, Jones RM, et al. A randomized trial of direct mailing of fecal occult blood tests to increase colorectal cancer screening. J Natl Cancer Inst. 2004;96:770–780. [PubMed]
27. Stokamer CL, Tenner CT, Chaudhuri J, Vazquez E, Bini EJ. Randomized controlled trial of the impact of intensive patient education on compliance with fecal occult blood testing. J Gen Intern Med. 2005;20:278–282. [PMC free article] [PubMed]
28. Sequist TD, Franz C, Ayanian JZ. Cost-effectiveness of patient mailings to promote colorectal cancer screening. Med Care. 2010;48:553–557. [PMC free article] [PubMed]
29. Kahn JS, Aulakh V, Bosworth A. What it takes: characteristics of the ideal personal health record. Health Aff (Millwood) 2009;28:369–376. [PubMed]
30. Halamka JD, Mandl KD, Tang PC. Early experiences with personal health records. J Am Med Inform Assoc. 2008;15:1–7. [PMC free article] [PubMed]
31. Hassol A, Walker JM, Kidder D, et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc. 2004;11:505–513. [PMC free article] [PubMed]
32. Chan EC, Vernon SW. Implementing an intervention to promote colon cancer screening through e-mail over the Internet: lessons learned from a pilot study. Med Care. 2008;46:S117–122. [PubMed]
33. Muller D, Logan J, Dorr D, Mosen D. The effectiveness of a secure email reminder system for colorectal cancer screening. AMIA Annu Symp Proc. 2009;2009:457–461. [PMC free article] [PubMed]
34. Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc. 2006;13:121–126. [PMC free article] [PubMed]
35. Fletcher RH, Lobb R, Bauer MR, et al. Screening patients with a family history of colorectal cancer. J Gen Intern Med. 2007;22:508–513. [PMC free article] [PubMed]
36. Arora NK, Hesse BW, Rimer BK, Viswanath K, Clayman ML, Croyle RT. Frustrated and confused: the American public rates its cancer-related information-seeking experiences. J Gen Intern Med. 2008;23:223–228. [PMC free article] [PubMed]
37. Hesse BW, Nelson DE, Kreps GL, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med. 2005;165:2618–2624. [PubMed]
38. Freedman AN, Slattery ML, Ballard-Barbash R, et al. Colorectal cancer risk prediction tool for white men and women without known susceptibility. J Clin Oncol. 2009;27:686–693. [PMC free article] [PubMed]
39. Park Y, Freedman AN, Gail MH, et al. Validation of a colorectal cancer risk prediction model among white patients age 50 years and older. J Clin Oncol. 2009;27:694–698. [PMC free article] [PubMed]
40. Waters EA, Sullivan HW, Nelson W, Hesse BW. What is my cancer risk? How internet-based cancer risk assessment tools communicate individualized risk estimates to the public: content analysis. J Med Internet Res. 2009;11:e33. [PMC free article] [PubMed]
41. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287:2691–2700. [PubMed]
42. Hsu J, Huang J, Kinsman J, et al. Use of e-Health services between 1999 and 2002: a growing digital divide. J Am Med Inform Assoc. 2005;12:164–171. [PMC free article] [PubMed]