Critical Issues in eHealth Research]]>Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal; ]]>
Am J Prev Med. Author manuscript; available in PMC 2009 August 3.
PMCID: PMC2720032

Critical Issues in eHealth Research


Since the early 1950s, computer technologies have been used to gather, manage, and disseminate health information.1 The development of microcomputer processing in the 1970s and its subsequent exponential increase in speed and memory, together with the proliferation of electronic network infrastructures to transmit large amounts of data (e.g., Internet, intranets, electronic medical records, telecommunication networks), have created ever-expanding opportunities to assess and deliver health information to individuals, groups, and populations. Computer technologies also have created extraordinary opportunities for real-time assessments of physiology (e.g., ambulatory heart rate monitoring, galvanic skin response); health behaviors (e.g., accelerometry); and self-reports (e.g., personal digital assistants [PDAs] that capture stress levels, diet, medication use) in the real-world and to deliver individualized information in response.23 In addition, interventionists can use the rapidly expanding palette of technologic capabilities to deliver evidence-based programs through the Internet in ways that are both individualized and cost-effective. There is evidence that behavior change programs delivered via the Internet, using data provided by users, can be highly effective.47

Yet, in spite of a surge of technologic capability, the corresponding research and evaluation methodologies have not kept pace with the rapid evolution and proliferation of health information and communication technologies. Nor has the dissemination of effective eHealth interventions achieved the level of penetration one might have hoped, given the number of people who now possess access to the Internet.


In June 2005, the National Institutes of Health (NIH), convened a scientific conference to provide an interdisciplinary forum to discuss major theoretical and methodologic issues in eHealth research — Critical Issues in eHealth Research. Prompted by the collective recognition that research methods should evolve along with advances in health information technologies, the conference convened government scientists, academic researchers from a variety of disciplines, computer engineers, patient advocates, and practitioners from the private and public sectors to examine the complex eHealth research issues related to the intersection among health science, communication technology, and research methodology. The conference was co-sponsored by the these divisions of NIH: National Cancer Institute, National Institute on Drug Abuse, National Institute of Mental Health, National Library of Medicine, Office of Behavioral and Social Sciences Research, and the Office of Disease Prevention, as well as the Health e-Technologies Initiative (a national program office of the Robert Wood Johnson Foundation).

Critical Issues in eHealth Research identified and explored areas of congruence and controversy related to the development, evaluation, and dissemination of eHealth research and the products of this research. Scientists also discussed the interdisciplinary crossroads that lay the foundation for health applications of information and communication technologies, focusing particularly on the juncture among behavioral science, basic biomedical science, computer engineering, and the information sciences.

Use of the Internet in health research received extensive attention at the inaugural Critical Issues in eHealth Research conference for good reason. Public adoption of the Internet soared from a mere 15% of Americans in 1995 to an estimated 75% of Americans in 2006,8 with more than 70 million American adults going online to use the Internet each day.9 As more Americans are online, they rely on the Internet for important health information.1011 A 2005 report from the Pew Foundation estimates that 80% of Internet users have searched online for health information at some point in their lives,12 and a large percentage of “health seekers” say the resources they find on the web have a direct effect on the decisions they make about their health care and on their interactions with doctors.13 Use of the Internet by Americans is just the tip of the iceberg, with projections for global use reflecting a true transformation in the ways in which governments address public health issues, businesses pursue economic opportunity, and individuals lead their lives.1415

With respect to disease prevention and control, increased use of the Internet can facilitate: information acquisition11, 16; tailored dissemination of health information5, 15, 17, 18; academic, commercial and governmental collaborations1820; the adoption of healthy lifestyles, such as changes in diet, physical activity, weight, and smoking47, 15, 17, 2224; detection and treatment of disease2527; and community outreach.21, 2830


The papers in this supplement3141 highlight the major presentations at the 2005 Critical Issues in eHealth Research conference. They bring a unique perspective to the literature on key methodologic issues surrounding eHealth research and suggest new research methods for evaluation of eHealth applications. Diverse definitions of “eHealth” have been advanced, with little consensus.42,43 For purposes of the conference, eHealth research was defined broadly as use of health information and communication technologies (e.g., Internet, personal digital assistants, digital cell phones, accelerometers, personalized/electronic health records, electronic medical records, CD-ROM) to assess, monitor, and improve health within a research context.

Ahern,31 Hesse and Schneiderman,32 and the invited commentary by Slack33 provide historical perspectives about eHealth and user interactions with health information technologies. Ahern outlines general research opportunities and challenges facing eHealth research and emphasizes the need to “promote adherence to the highest standards of research design and methodologic rigor to improve the overall quality of eHealth research.”31 Ahern also highlights the importance of theory in the development of eHealth research.

Hesse and Schneiderman32 observe that potential errors in eHealth research and health care can be reduced and/or avoided and program effectiveness maximized by incorporating users’ perspectives (how individuals interact with technologies and healthcare systems) into eHealth research and intervention design. They offer an ecologic conceptual framework that depicts the multi-level nature of the healthcare environment and suggest that eHealth programs consider the goals of users and health systems. Further, they highlight some of the gains that could be achieved in development of user-centered eHealth interventions. Such approaches would transcend focus groups and other formative research methods traditionally used in health behavior research. Instead, the designs of interventions and tools would be guided by patients’ and physicians’ needs rather than by a priori or imposed models of need. Increasingly, the value of this approach is appreciated in the business sector.44

Couper34 systematically addresses the specific (and thorny) research issues of sample representation in Internet surveys. This topic illustrates one of the key threats to the validity and generalizability of eHealth research findings. Dillman and Smyth35 complement Couper’s discussion of Internet surveys by focusing on the relationship between visual design of Internet surveys and measurement error. Guidelines for Internet survey design are provided to increase standardization of surveys and reduce measurement error.

Lobach and Detmer36 discuss the methodologic challenges of a separate eHealth research tradition, namely electronic health records (EHRs; also referred to as electronic medical records [EMRs]). In contrast to the Internet, EHRs have not been as readily adopted by medical practices and health systems as their apparent utility might lead us to expect. Unique methodologic challenges face evaluation of EHRs. Lobach and Detmer observe that traditional randomized controlled research designs may not be feasible in research using EHRs, and they discuss alternative research designs. Similar to the view put forth by Hesse and Schneiderman,32 Lobach and Detmer suggest that consideration of the multiple levels of the healthcare system and the human–computer interface are vital to obtain reliable and valid health information. They also echo the recommendation of Ahern31 to incorporate the research question (i.e., theory) into the design of eHealth studies, in this case, using EHR evaluation.

Collins and colleagues37 offer two novel methods for developing and evaluating eHealth interventions. The methods proposed, based on randomized experiments, provide researchers ways to maximize and tailor behavioral interventions using the richness of electronic data. Moreover, the methods Collins proposes, which parallel trends in the evaluation of clinical trials, are aimed at getting research answers more expeditiously and efficiently. These innovative methods take advantage of the extensive data that can readily be collected in eHealth research.

Glasgow38 extends the discussion of eHealth research beyond assessment and intervention to research dissemination. The concept of practical eHealth trials is offered in which study representation, research settings, alternative comparison groups, and relevant stakeholder outcomes all are considered in an effort to maximize balance between internal validity and generalizability of eHealth study results. Glasgow provides research recommendations and a discussion about the RE-AIM dissemination model as potential strategies by which to extend eHealth research into real-world practice. Consideration of how to disseminate effective eHealth interventions is critical since it is clear from other health domains that dissemination usually falls short of its potential. Because of the possibility of scaling eHealth interventions for dissemination to many millions of people, it is critical that we understand how to achieve dissemination of effective eHealth interventions that is appropriate and efficacious, not just pervasive.

Common themes among the various papers in this special issue include the importance of examining sample representation, tailoring information according to users’ needs and characteristics, considering multiple levels of analyses (e.g., individual, dyad, environment, policy, systems), and using theoretical models/research questions to guide the development of study designs. In addition, eHealth research among at-risk and underserved populations, health disparities, digital divide, privacy/informed consent, and economic issues were discussed and debated at the 2005 Critical Issues in eHealth Research conference. While a full discussion of these topics is beyond the scope of this paper, invited commentaries by Curry39 and Viswanath and Kreuter40 touch on several of the critical issues in these areas.


eHealth research is at an early stage of development. The 2005 Critical Issues in eHealth Research conference and subsequent papers in this special issue represent notable benchmarks that future eHealth research studies can reference. eHealth research and the information collected from such research are complex. If designed, developed, and employed optimally, eHealth applications have the potential to bridge health disparities, facilitate analysis of population level data to enable tailoring of healthcare delivery, and speed the translation of discoveries into practice—advances that parallel revolutionary developments in biomedical science in the form of molecular targeting of drugs and the mapping of the human genomes.

Like other biomedical advances, eHealth applications have the potential for both benefits and harms. The latter include the potential to endanger patient privacy, increase health disparities, lead patients away from effective treatments and widen the digital divide. The latent power of this burgeoning medium requires researchers from different sectors (industry, government, and academia) to collaborate on how best to harness the technical capabilities of emerging information technologies to support the social and cultural realities in which people work and live, while enhancing the systems ability to address the health needs of individuals. Only in this way will research methodologies emerge that effectively support, guide and direct these applications. In a global world in which national boundaries no longer constrain information flow, it behooves us to use all the tools at our command to improve health across the world. Increasingly, eHealth tools are an important part of the arsenal for improving population health. Finally, over the years, considerable concern has been raised about the possibility that computers will eclipse people. A strong message delivered by multiple speakers at the conference was about the centrality of the human interface, perhaps best summarized by Hesse and Schneiderman32: Patients and their families must be engineered into the system as a first line of defense against potential errors, not the last.

We should also heed the caution articulated by Crounse41 in his invited commentary that: all too often our methodology falls back on that which we know and have always done before....But we must...not dig in our heels, resist change and continue to conduct business as we’ve always done so before just because it suits our comfort level. Others around the world will not indulge in, or tolerate that luxury.


No financial conflict of interest was reported by the authors of this paper.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.


1. Shortliffe EH, Blois MS. The computer meets medicine and biology: Emergence of a discipline. In: Shortliffe EH, Perrault LE, Widerhold G, Fagan LM, editors. Medical informatics: Computer applications in health care and biomedicine. 2nd edition. New York: Springer; 2001. pp. 3–40.
2. Stone AA, Shiffman S, Atienza AA, Nebeling L. The science of real-time data capture: Self-reports in health research. Oxford: New York; 2007.
3. Patrick K, Intille SS, Zabinsky MF. An ecological framework for cancer communication: Implications for research. J Med Internet Res. 2005;7(3):1–7. [PMC free article] [PubMed]
4. Tate DF, Jackvony EH, Wing RR. Effects of Internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trial. JAMA. 2003;289(14):1833–1836. [PubMed]
5. Tate DF, Jackvony EH, Wing RR. A self-regulation program for maintenance of weight loss. N Engl J Med. 2006 Oct 12;355(15):1563–1571. [PubMed]
6. Graham AL, Abrams DB. Reducing the cancer burden of lifestyle factors: opportunities and challenges of the Internet. J Med Internet Res. 2005;7(3):e26. [PMC free article] [PubMed]
7. Cobb NK, Graham AL, Bock BC, Papandonatos G, Abrams DB. Initial evaluation of a real-world Internet smoking cessation system. Nicotine Tob Res. 2005 Apr;7(2):207–216. [PMC free article] [PubMed]
8. Pew Foundation. Percentage of U.S. [Accessed 2/5/2007]. Adults Online
9. Horrigan J, Rainie L. A decade of adoption: How the Internet has woven itself into American life. Pew Internet and American Life Project. [1/25/2005].
10. Madden M, Fox S. Finding Answers Online in Sickness and in Health. Pew Internet and American Life Project. [May 2, 2006].
11. Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK, Rimer BK, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med. 2005;165(22):2618–2624. [PubMed]
12. Fox S. Online Health Search 2006. 2005 Pew Internet and American Life Project. [Accessed Dec. 4, 2006].
13. Madden M, Fox S. Finding Answers Online in Sickness and in Health. 2005. Pew Internet Report.
14. Friedman TL. The world is flat: A brief history of the twenty-first century. New York: Farrar, Straus and Giroux; 2006.
15. Spittaels H, De Bourdeaudhuij I. Implementation of an online tailored physical activity intervention for adults in Belgium. Health Promot Int. 2006;21(4):311–319. [PubMed]
16. Baker L, Wagner TH, Singer S, Bundorf MK. Use of the Internet and e-mail for health care information: results from a national survey. JAMA. 2003;289(18):2400–2406. [PubMed]
17. Buller DB, Borland R, Woodall WG, Hall JR, Hines JM, Burris-Woodall P, et al. Randomized Trials on Consider This, a Tailored, Internet-Delivered Smoking Prevention Program for Adolescents. Health Educ Behav. in press. [PubMed]
18. Atkinson NL, Gold RS. Online research to guide knowledge management planning. Health Educ Res. 2001;16(6):747–763. [PubMed]
19. Glanz K, Hoelscher D. Increasing fruit and vegetable intake by changing environments, policy and pricing: restaurant-based research, strategies, and recommendations. Prev Med. 2004;39 Suppl 2:S88–S93. [PubMed]
20. Nakajima K, Kurata Y, Takeda H. A web-based incident reporting system and multidisciplinary collaborative projects for patient safety in a Japanese hospital. Qual Saf Health Care. 2005;14(2):123–129. [PMC free article] [PubMed]
21. Rimer BK, Conaway M, Lyna P, Glassman B, Yarnall KS, Lipkus I, et al. The impact of tailored interventions on a community health center population. Patient Educ Couns. 1999;37(2):125–140. [PubMed]
22. Anhoj J, Jensen AH. Using the Internet for life style changes in diet and physical activity: a feasibility study. J Med Internet Res. 2004;6(3):e28. [PMC free article] [PubMed]
23. Noah SA, Abdullah SN, Shahar S, Abdul-Hamid H, Khairudin N, Yusoff M, et al. DietPal: a Web-based dietary menu-generating and management system. J Med Internet Res. 2004;6(1):e4. [PMC free article] [PubMed]
24. Richardson CR, Brown BB, Foley S, Dial KS, Lowery JC. Feasibility of adding enhanced pedometer feedback to nutritional counseling for weight loss. J Med Internet Res. 2005;7(5):e56. [PMC free article] [PubMed]
25. Frank MS. Opinion: a prototype for a computerized national mammography registry and tracking system using telecommunications and the Internet. J Digit Imaging. 1995;8(3):127–131. [PubMed]
26. Tamm EP, Raval BK, Huynh PT. Evaluation of the quality of self-education mammography material available for patients on the Internet. Acad Radiol. 2000;7(3):137–141. [PubMed]
27. Thomas B, Stamler LL, Lafreniere KD, Out J, Delahunt TD. Using the Internet to identify women's sources of breast health education and screening. Women Health. 2002;36(1):33–48. [PubMed]
28. Huber JT, Walsh TJ, Varman B. Camp for all connection: a community health information outreach project. J Med Libr Assoc. 2005;93(3):348–352. [PMC free article] [PubMed]
29. Spatz MA. Providing consumer health information in the rural setting: Planetree Health Resource Center's approach. Bull Med Libr Assoc. 2000;88(4):382–388. [PMC free article] [PubMed]
30. Stromgren C. Community outreach through Web resources: multiple sclerosis. Med Ref Serv Q. 2002;21(1):61–70. [PubMed]
31. Ahern DK. Challenges and Opportunities in eHealth Research. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
32. Hesse BW, Shneiderman B. eHealth Research from the User's Perspective. Am J Prev Med. 2007;32:SXX–SXX. [PMC free article] [PubMed]
33. Slack W. Cybermedicine for the Patient. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
34. Couper MP. Issues of Representation in eHealth Research (with a Focus on Web Surveys) Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
35. Dillman DA, Smyth JD. Design Effects in the Transition to Web-Based Surveys. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
36. Lobach DF, Detmer DE. Research Challenges for Electronic Health Records. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
37. Collins LM, Murphy SA, Strecher V. The Multiphase Optimization Strategy (MOST) and the Sequential Multiple Assignment Randomized Trial (SMART): New Methods for More Potent eHealth Interventions. Am J Prev Med. 2007;32:SXX–SXX. [PMC free article] [PubMed]
38. Glasgow RE. eHealth Evaluation and Dissemination Research. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
39. Curry SJ. eHealth Research and Healthcare Delivery: Beyond Intervention Effectiveness. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
40. Viswanath K, Kreuter MW. Health Disparities, Communication Inequalities and E-Health A Commentary. Am J Prev Med. 2007;32:SXX–SXX. [PMC free article] [PubMed]
41. Crounse B. The newspaper, the wristwatch, and the clinician. Am J Prev Med. 2007;32:SXX–SXX. [PubMed]
42. Eysenbach G. What is e-health? J Med Internet Res. 2001. p. e20. 4–5. [PMC free article] [PubMed]
43. Oh H, Rizo C, Enkin M, Jadad A. What is eHealth?: A systematic review of published definitions. World Hosp Health Serv. 2005;41(1):32–40. [PubMed]
44. Kelly T, Littman J. The art of innovation 2001. New York: Doubleday;