eHealth Research from the User’s Perspective
1 National Cancer Institute, Bethesda, Maryland
2 University of Maryland, College Park, Maryland
The application of Information Technology (IT) to issues of healthcare delivery has had a long and tortuous history in the U.S. Within the field of eHealth, vanguard applications of advanced computing techniques, such as applications in artificial intelligence or expert systems, have languished in spite of a track record of scholarly publication and decisional accuracy. The problem is one of purpose, of asking the right questions for the science to solve. Historically, many computer science pioneers have been tempted to ask “what can the computer do?” New advances in eHealth are prompting developers to ask “what can people do?” How can eHealth take part in national goals for healthcare reform to empower relationships between healthcare professionals and patients, healthcare teams and families, and hospitals and communities to improve health equitably throughout the population? To do this, eHealth researchers must combine best evidence from the user sciences (human factors engineering, human-computer interaction, psychology, and usability) with best evidence in medicine to create transformational improvements in the quality of care that medicine offers. These improvements should follow recommendations from the Institute of Medicine to create a health care system that is (a) safe, (b) effective (evidence-based), (c) patient-centered, and (d) timely. Relying on the eHealth researcher’s intuitive grasp of systems issues, improvements should be made with considerations of users and beneficiaries at the individual (patient/physician), group (family/staff), community, and broad environmental levels.
The application of information technology (IT) to issues of personal health and healthcare has had a long and surprisingly complex history within the United States. It was not long after the first mainframe computer systems began automating record keeping that hospitals began applying IT to the tasks of billing and financial record keeping.1
Yet, in spite of an early application of IT to administrative and actuarial tasks, the challenge of developing a consistent, interoperable system for transporting patients’ medical records across health care systems has been an elusive and seemingly intractable goal.2
Reasons given for delays in this and other areas of health IT include grappling with competing incentive structures, dealing with issues of standardization, meshing with time-pressured workflows, ensuring privacy, and creating systems that are useful and usable to data producers as well as data users.3–6
In short, reasons given have not focused as much on inadequacies of the technology, but more on the interface between the social and the technical, between users and the system.7
ASKING WHAT COMPUTERS CAN DO
In the same vein, the field of medical informatics has produced some of the most advanced computing technologies to be applied in real world settings. Applications of artificial intelligence to medicine in the 1980s produced such vanguard systems as the rule based MYCIN expert system at Stanford University8
, the Iliad,9
resources at the University of Utah, and the Quick Medical Reference (QMR)11
physician aid at the University of Pittsburgh. Yet, after years of successful testing and scholarly publication many of the most sophisticated diagnostic systems — at least as originally scoped — have fallen into disuse.1, 12
Users reported feeling uncomfortable relying on the decisional recommendations of an opaque, black box computer system no matter how much more accurate the system’s decisions were compared to those of actual clinicians.12
In many respects, the hope and hype of IT applied to healthcare reflect the lessons learned from computer science more generally. The field of computer science is littered with applications that, although technologically superior, have failed to find a following among discriminating users. From an historical perspective, the problem has been one of purpose and focus; of asking the right question for the science to solve. The field of artificial intelligence is a good example. Early applications followed a model of automating human processes; that is, of creating applications that mimicked what humans do. The motivation stemmed in part from an industrial view of technology in which machinery was engineered with the explicit purpose of replacing the physical labor of factory workers, thereby saving money through reductions in personnel costs.13
It also stemmed from an unabashed, romantic notion of creating a future inhabited by smart, thinking machines.14
Quite simply, the science of artificial intelligence often found itself following the art of science fiction.15
Later applications, the ones that survived, changed their focus. Rather than supplant human capabilities, the newer applications were focused on supplementing or complementing the information processing abilities of their operators.16
The lexical applications embedded in many popular word processing systems are a good example. These systems run unobtrusively in the background but offer a comfortable safety net to writers in checking for spelling errors, monitoring for punctuation, and offering suggestions for grammatical improvements. In a similar way, reminder systems in hospitals can use advanced computing techniques to keep a patient’s therapeutic regimen on track17
or to protect healthcare professionals from prescribing medications with documented interactions.18
As one commentator put it, these new applications serve to “informate” rather than “automate” human processes. They add value precisely because their routines improve on human performance, and do not try to replace it.13
ASKING WHAT PEOPLE CAN DO
The field of eHealth research — as with research in other vital areas of computer application —is itself striving to achieve a new stage of maturation. During the pioneering days of eHealth the question was often about what the computer
could do. During the next phase, the crucial question to ask is: what can people
do? How can advanced computing and telecommunications be used in conjunction with existing systems of care to ensure that people live longer and higher quality lives? David Brailer, former National Health Information Technology Coordinator for the U.S. Department of Health and Human Services, said it this way. “Everyone thought IT was about computers, but we’ve refined that to say that IT is about healthcare — it’s about the experience we really have.”19
THE EHEALTH EXPERIENCE
The notion of eHealth gained currency in the 1990s at a time when public expansion of the Internet promised to transform information exchange, product acquisition, and service delivery. Research on the benefits of using wide area network technologies, at least among scientists and technical administrators, had suggested that distributed computing could bring people and resources together in ways that had not been seen before.20, 21
eHealth, like eCommerce and eGovernment, promised to transform the status quo by breaking down barriers of time and place. This mix of technology, as well as the entrepreneurial and idealistic spirit of taking benefits directly to consumers, remains inseparably mixed in definitions of eHealth today.22, 23
For medical informaticians and policy makers, the promise of eHealth opened up a capacity to link advances in medical information technology to the advantages promised by an emerging national health information infrastructure.24
This led to a number of pioneering efforts to bring advanced technologies, incubated previously within hospital settings, onto the web for patients and practitioners to use. Complex decision support technologies were mounted on the web for patients’ use. Large scale bibliographic databases were pushed to the public.
Reactions from the public to this push were mixed, and did not conform neatly to expectations.25
In 1997, when speculation was high and innovation outpaced demand, only 22.2% of the US population aged 3 and older reported having access to the Internet.26
By 2003, after the “dot com bust” was predicted to dampen consumer enthusiasm, Internet penetration soared to the point at which an estimated 63% of adults 18 years and older were online27, 28
. By 2005, the Internet had become part of the “new normal;” it had become part of the constellation of everyday media and resources people use as supports in their ongoing busy lives.29
As early as 1999, surveys had revealed that health information seeking was a frequent and common practice on the web.30
What people often found online, though, was a cacophony of mixed messages and information sites of unequal quality and usefulness. Visitors to commercial sites were wary of ulterior motives and unfounded claims31, 32
; visitors to government and academic sites were inundated with highly technical language and raw data with little synthesis or integration.33, 34
In an age of abundant health information, consumers reported feeling confused and perplexed.35
Estimates suggested that in 2003 more than three quarters of the general adult population would endorse the statement that there were too many prevention recommendations in the area of cancer to know which ones to follow.36
About half of the population reported looking for information on cancer either for themselves or for others, and out of that group the Internet was the first place they looked. At the same time, a majority of Americans reported having greater confidence in their physicians than in the Internet.27
Newer results indicate that the credibility differences between physicians and the Internet may be widening, with trust in physicians trending upwards and trust in the Internet trending downwards.37
HEALTHCARE IN THE INFORMATION AGE
While the strain of keeping up with the explosion of information on the web is challenging for consumers, the task of staying current on the tsunami of mounting medical information is even more challenging for healthcare professionals. In 2004, the National Library Medicine reported that it added more than half a million (571,000) new references to its online bibliographic database.38
By some estimates, a general internist would have to read 20 articles a day for 365 days a year just to keep current on relevant medical knowledge.39
Added to the burden of keeping current on research is the strain of keeping up with an increasingly mobile patient base. The goal of maintaining a “life span” perspective on a patient’s health19
is complicated by the challenge of keeping records current on a lifetime’s worth of medical history, laboratory results, specialists’ reports, pharmaceutical records, adverse reactions, and psychosocial indicators.40
Indeed, combining the best of current medical knowledge with a patient’s own historical data in an age of “personalized medicine,”41
will be next to impossible without the assistance of health information technology. Without an electronic infrastructure to guarantee safekeeping and backup of patients’ data, it is impossible to guarantee continuity of care for some of the most vulnerable in the population42
— a crisis acutely manifest in the 2005 Hurricane Katrina disaster.43
As medical technology advances and the health information environment becomes more complex, healthcare becomes more vulnerable to error. Health systems analysts have estimated that up to 98,000 avoidable deaths per year occur because of a lack of supportive safety mechanisms in pharmacy, surgery, hospital care, and outpatient care.40, 44
An analysis of medical charts for women presenting with late state cervical cancer within one medical system revealed that more than half of the deaths could have been avoided if the system had kept women up-to-date on routine pap screens.45
The good news from the patient’s perspective is that advances in biomedical science in the 20th
Century may have nearly doubled life expectancy; the bad news is that the system of healthcare has grown complex, reactive, and potentially dangerous.46
Clearly health information technologies will, and must, be part of the solution to guarantee that safeguards are engineered into the environment in which healthcare takes place.40, 47, 48
It must be remembered, however, that the science of informatics is the science of sociotechnical systems49
; the science of “people, tools, and conversations” working in synchrony to achieve common goals.7
eHealth research must become part of the system of reform that will link evidence-based medical practices to national standards of equity, safety, usability, and patient-centeredness. It must work in tandem with physicians, nursing staff, office staff, and other links in the healthcare chain to comprise a safety net of support for patients.
USERS IN AN EHEALTH ENABLED SYSTEM
The Institute of Medicine (IOM) presented a framework for healthcare reform in its report titled: Crossing the Quality Chasm: A New Healthcare System for the 21st Century40
. In , we take the users who are described in that report and present them graphically as two interlocking sets of concentric circles. At the center of the figure is the direct interaction between the healthcare professional and each individual patient. This is the nexus of the healthcare relationship and symbolizes the direct point of accountability between an individual’s own self-management and the healthcare provider’s training and recommendations.
Users within the healthcare system as described in the Institute of Medicine’s prescriptive framework for health system reform.
Surrounding the provider-patient link are concentric circles of expanding group and systems support. On the healthcare provider side, the individual practitioner works as one member of a coordinated “micro-unit” of care: office staff, surgical teams, long term care teams, and so on. The micro-unit is in turn embedded within larger enabling systems: hospitals, health maintenance organizations, preferred provider organizations, and others. On the public side, the patient is surrounded by family, friends, and neighbors as the most immediate circle of support. Public interactions occur within the greater sphere of a functioning community, with churches, businesses, civic groups, and social services. Both sides of the relationship are in turn embedded within a larger context of policies, regulations, and incentives.
TASKS, GOALS, AND OBJECTIVES
In , we juxtapose types of users against the major tasks, goals and objectives outlined in the IOM’s framework.50
We have selected four overarching goals as being especially relevant to a discussion of eHealth and have listed them as labels for the four columns in the table. At the junction of row and column we indicate how the goals and objectives of the new healthcare framework become relevant across levels of users. The individual cells represent the particular “use case scenarios” that the eHealth-enabled system must support.
Health system goals as a function of level of user
A core tenet of the IOM’s prescriptive framework is that safety must be made an inherent attribute of the system.40, 50
If information is too complex to understand, especially under periods of duress or high cognitive load, then the interface to that information must be simplified. If critical events fail to occur because of lapses in memory, then the system must monitor for completion of those events and send status reminders as appropriate to decision makers until they are fulfilled or dismissed. The principle is to make the healthcare system foolproof by designing the interface between components to be consistent, predictable, and controllable by all of its users.51
Throughout the first column in , we emphasize the theme of safety as a system property for eHealth across levels of users. On the delivery side, a coordinated system of safeguards should allow teams of healthcare professionals to work interdependently in a system that promotes safe practices and encourages continuous quality improvement. Informatics will offer a seamless level of information support on this side, but the technologies should be introduced into the system only after following a rigorous schedule of iterative usability testing and formative evaluation.52
“Human centered” testing methods applied within the context of “bench to bedside” research should be as important to the informatics enterprise as Randomized Clinical Trials are for the introduction of new pharmaceutical agents.53
On the public side, self-empowered patients and their caregivers must be considered as crucial elements of care in the overall system. Day-to-day adherence to medical regimens takes place in the home; self-monitoring for early symptoms takes place at a personal level; and the preponderance of crucial and vital decision making takes place between family members.54–59
It is within these personal patient spheres that healthcare actually begins and often ends. Every tool and every connective network meant to support these spheres must be engineered with the same strict set of human factors standards and testing that will govern the development of safe systems within hospital settings. This is currently not the case. Something as simple as the computerized label printed on a pill bottle is usually so jargon-laden, and so functionally imprecise, that patients are at serious risk for error in the home.60
Patients and their families must be engineered into the system as a first line of defense against potential errors, not the last58
Medicine extended the life span of individuals in the 20th century by moving from a practice of art based on speculation, to a practice of science based on evidence. Bringing the science and evidence of medicine into the care process at the right times is another high priority challenge in the IOM’s prescription.
Throughout the course of the 20th
century the assumption behind medical training was that the best way to transmit knowledge from its source to its use in patient care was “to first load the knowledge into human minds … and then expect those minds, at great expense, to apply the knowledge to those who need it.”61
Designing from the user’s perspective takes a different orientation62
. It embraces the notion that effective decision making is a product of knowledge contained in the mind as well as knowledge stored in the environment.63
Taking this perspective, the emphasis shifts from personal heroics to creating a system of knowledge to guide evidence-based delivery, a shift that has been shown to yield significant improvements in patient outcomes and reductions in costs.64, 65
Another area of productive research is the translation of nonlinear, high density information fields into meaningful displays that can be quickly assimilated through visual inspection.66, 67
Work being conducted in the Human-Computer Interaction Laboratory at the University of Maryland is producing a new look at the task of combining all of the relevant information needed for a clinical encounter into single, highly effective visual displays.66, 68
Work being funded by the National Cancer Institute is enabling comparable approaches for displaying personal risk information to patients visually, with significant yields in comprehension and usefulness.69–71
With all the discussion of multiple users, from physicians to care units to hospitals, and from patients to caregivers, the ultimate goal of the healthcare system is to improve patient outcomes. All of the tools, conversations, and decisions that take place within a healthcare system must be measured against the benchmark of patient need in order to achieve system-wide objectives of safety and effectiveness.54, 55, 59
One aspect of patient-centeredness is customized care.72
Advances in biomarker detection73
, targeted pharmacogenetics74
, and tailored communication75
are making it possible to focus care individually on the specific needs of each patient. EHealth applications will be part of this effort as they assist in providing the right information, to the right person, at the right time.76
Caution must be taken to incorporate evidence from the existing literature on human-computer interaction when doing this. Many theoretically elegant interfaces (adaptive interfaces, social agents, expert systems) have failed in real world settings. When measured against the benchmark of user goals, the systems were perceived as being unpredictable and inconsistent. 15
Another aspect of patient-centeredness, call this population-centeredness, is the goal of extending the reach of medical knowledge to serve a diversity of beneficiaries in an equitable fashion. This means that the public interface to medicine, from instructions on a prescription bottle to a web-based portal for access to personal health information, must be equitably accessible to all users. One strategy for accomplishing that goal in eHealth is to adopt the principle of Universal Design.15, 77
Universal design is the principle of creating a structured environment to be equally accessible by all users, regardless of experience level or physical ability. A simple example in the physical world is the presence of “curb cuts” at the entrance to stores and office buildings, or the inclined ramps cut into sidewalks that provide easy passage for those in wheelchairs as well as anyone wheeling a shopping cart or set of heavy luggage.
Curb cuts do not just work in physical environment, but with a little forethought can be built into the virtual eHealth environment.78
Adherence to the standards set by section 508 of the Rehabilitation Act (see http://www.access-board.gov/508.htm
) will ensure that websites are compliant with text-reading software for users who are visually impaired and will not get in the way of a sighted user’s ability to interact with the site.79
Likewise, following prescriptions from the “Plain Language” movement will help ensure that labels, instructions, and health-related content in eHealth applications are equally accessible to all users regardless of educational background or medical experience.80, 81
Guidelines documents82, 83
as well as automated assessment tools84
are readily available to assist the eHealth designer in making applications universally usable by a diversity of audiences. Formative usability testing, conducted across the full range of expected user types, will also help ensure that the final product is equally accessible to a broad range of users.83
It has been said that time is the currency of the information age. Yet, one of the “problems with computers”85
is a failure to account for the accumulated effect of small temporal disruptions on the overall performance of human systems.85–87
Consider the cumulative impact of dealing with pop-up ads on the web, or spending an inordinate amount of time being shuffled from one multilayered telephone answering system to another before being connected to an advice nurse at a hospital. The error is one of myopia, argued Zuboff and Maxmin,88
as engineers focus solutions on single transactions without considering the impact of multiple interactions on trust and relationships.
The IOM emphasized that a patient-centered healthcare system must move away from a transaction-based management environment, to an environment that fosters “healing relationships” over time. To do this, all components of the system must be coordinated to provide levels of “deep support”88
for patients and their caregivers. Coordinated eHealth technologies can and should be developed to keep track of the individual needs of patients, and to ensure that all the relevant members of the healthcare team, including the patients themselves, are notified in a timely manner. With an interoperable system of electronic medical records in place65
, eHealth applications can be built to encourage prevention and health maintenance, and to catch potential treatment errors early before any true harm is done.89–91
We are at a turning point in eHealth research. Taken down the right path –if we ask the right question– eHealth will become part of the solution in turning the tide of medical errors and extending quality of life equitably throughout the population. Taken down the wrong path –if we ask the wrong questions– eHealth will escalate the crisis in healthcare as only computers can do.85
Fortunately, there is a way through the conundrum. The path lies in merging the science of evidence-based medicine with the practice of user-centered research. This is a transdisciplinary92
union, bringing the knowledge bases of a rapidly advancing biomedical research enterprise into synchrony with a burgeoning evidence base in the cognitive and computational sciences.15, 83, 93
The union is needed now more than ever, and cannot be taken for granted. Case studies from the diffusion of innovation highlight what happens when less-than-optimal designs catch the “tipping point”94
of diffusion before they have been optimized for user needs and constraints. The results, in the most benign sense, can be an ascendancy of mediocrity (consider the seemingly intractable diffusion of the QWERTY keyboard); results in the most severe case can cost lives. By riveting technology development on the questions that matter, eHealth researchers can provide a feedback loop into health system reform. More importantly, eHealth researchers can help to accelerate progress for improving lives; the metric that matters most from the user’s perspective.
This literature review was funded by the National Cancer Institute.
No financial conflict of interest was reported by the authors of this paper.
1. Shortliffe EH. Medical informatics : computer applications in health care and biomedicine. 2. New York: Springer; 2001.
2. Shortliffe EH. Strategic action in health information technology: why the obvious has taken so long. Today the United States is poised to achieve what has been sought and anticipated for at least three decades. Health Aff (Millwood) 2005;24(5):1222–33. [PubMed] 3. Wears RL, Berg M. Computer technology and clinical work: still waiting for Godot. Jama. 2005;293(10):1261–3. [PubMed]
4. Malhotra Y, Galletta DF. Building systems that users want to use. Commun ACM. 2004;47(12):88–94.
5. Heeks R. Health information systems: Failure, success and improvisation. Int J Med Inform. 2006;75(2):125–37. [PubMed] 6. Kaplan B. Addressing organizational issues into the evaluation of medical systems. J Am Med Inform Assoc. 1997;4(2):94–101. [PMC free article] [PubMed] 7. Coiera E. Four rules for the reinvention of health care. Bmj. 2004;328(7449):1197–9. [PMC free article] [PubMed]
8. Buchanan BG, Shortliffe EH. Rule-based expert systems : the MYCIN experiments of the Stanford Heuristic Programming Project. Reading, Mass: Addison-Wesley; 1984.
9. Warner HR, Jr, Bouhaddou O. Innovation review: Iliad--a medical diagnostic support program. Top Health Inf Manage. 1994;14(4):51–8. [PubMed] 10. Haug PJ, Gardner RM, Tate KE, Evans RS, East TD, Kuperman G, et al. Decision support in medicine: examples from the HELP system. Comput Biomed Res. 1994;27(5):396–418. [PubMed] 11. Miller R, Masarie FE, Myers JD. Quick medical reference (QMR) for diagnostic assistance. MD Comput. 1986;3(5):34–48. [PubMed] 12. Miller RA. Medical diagnostic decision support systems--past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc. 1994;1(1):8–27. [PMC free article] [PubMed]
13. Zuboff S. In the age of the smart machine : the future of work and power. New York: Basic Books; 1988.
14. Stork DG, editor. HAL’s Legacy: 2001’s Computer as Dream and Reality. Cambridge, MA: MIT Press; 1997.
15. Shneiderman B. Leonardo’s laptop : human needs and the new computing technologies. Cambridge, Mass.: MIT Press; 2002.
16. Hesse BW. Harnessing the power of an intelligent health environment in cancer control. Stud Health Technol Inform. 2005;118:159–76. [PubMed] 17. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med. 2000;160(3):301–8. [PubMed]
18. Kuperman GJ, Bobb AM, Payne T, Avery AJ, Gandhi TK, Burns G, et al. Medication-related Clinical Decision Support in Computerized Provider Order Entry Systems: A Review. J Am Med Inform Assoc. 2006
19. Brailer D. Action through collaboration: a conversation with David Brailer. The national coordinator of HIT believes that facilitation, not mandates, are the way to move the agenda forward. Interview by Robert Cunningham Health Aff (Millwood) 2005;24(5):1150–7.
20. Hesse BW, Sproull L, Kiesler SB, Walsh JP. Returns to science: computer networks in oceanography. Commun ACM. 1993;36(8):90–101.
21. Sproull L, Kiesler S. Connections : new ways of working in the networked organization. Cambridge, Mass.: MIT Press; 1991.
22. Oh H, Rizo C, Enkin M, Jadad A. What is eHealth?: a systematic review of published definitions. World Hosp Health Serv. 2005;41(1):32–40. [PubMed] 24. Yasnoff WA, Humphreys BL, Overhage JM, Detmer DE, Brennan PF, Morris RW, et al. A consensus action agenda for achieving the national health information infrastructure. J Am Med Inform Assoc. 2004;11(4):332–8. [PMC free article] [PubMed] 25. Eysenbach G, Kummervold PE. “Is Cybermedicine Killing You?”-The story of a Cochrane disaster. J Med Internet Res. 2005;7(2):e21. [PMC free article] [PubMed]
26. Promotion OoDPaH., editor. U.S. Department of Health and Human Services. Communicating Health: Priorities and Strategies for Progress. U.S. Department of Health and Human Services; 2003.
27. Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK, Rimer BK, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med. 2005;165(22):2618–24. [PubMed]
28. Madden M. Online pursuits: the changing picture of who’s online and what they do. Washington, DC: Pew Research Center; 2003.
29. Pew Trends 2005: Information for the Public Interest. Washington, DC: Pew Research Center; 2005.
30. Rice RE. The Internet and health communication: A framework of experiences. In: Rice RE, Katz JE, editors. The Internet and Health Communication: Experiences and Expectations. Thousand Oaks, CA: Sage; 2001. pp. 5–46.
31. Gustafson DH, Robinson TN, Ansley D, Adler L, Brennan PF. Consumers and evaluation of interactive health communication applications. The Science Panel on Interactive Communication and Health. Am J Prev Med. 1999;16(1):23–9. [PubMed]
32. Eng TR, Gustafson DH. Wired for health and well-being : the emergence of interactive health communication. Washington, D.C.: Office of Disease Prevention and Health Promotion. U.S. Dept. of Health and Human Services; 1999. Science Panel on Interactive Communication and Health., United States. Dept. of Health and Human Services. Office of Disease Prevention and Health Promotion.
33. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. Jama. 2001;285(20):2612–21. [PubMed]
34. Shenk D. Data smog : surviving the information glut. 1. San Francisco, Calif.: Harper Edge; 1997.
35. Arora NK, Hesse BW, Rimer BK, Viswanath K, Clayman M, Croyle RT. JNCI. 2006. The Informed Patient: Struggling to Stay Afloat in a Sea of Cancer-related Information.
36. Han P. Perceived ambiguity about cancer prevention recommendations: Relationship to perceptions of cancer preventability, risk, and worry. J Health Commun. in press.
37. Hesse BW. Identifying Consumer Needs for Health Information. MedNet; Toronto, Canada: 2006.
38. National Library of Medicine. Medline. Bethesda, MD: National Institutes of Health; 2005. Fact Sheet .
39. Shaneyfelt TM. Building bridges to quality. Jama. 2001;286(20):2600–1. [PubMed]
40. Institute of Medicine (U.S.) Crossing the quality chasm : a new health system for the 21st century. Washington, D.C.: National Academy Press; 2001. Committee on Quality of Health Care in America.
41. Culliton BJ. Extracting Knowledge From Science: A Conversation With Elias Zerhouni. Health Aff (Millwood) 2006
42. Hewitt ME, Greenfield S, Stovall E. National Cancer Policy Board (U.S.) From cancer patient to cancer survivor : lost in transition. Washington, D.C.: National Academies Press; 2006. Committee on Cancer Survivorship: Improving Care and Quality of Life.
43. Conn J. IT systems survive Katrina. Hospitals with EMRs able to retrieve patient data. Mod Healthc. 2005;35(37):12–4.
44. Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? Jama. 2005;293(19):2384–90. [PubMed] 45. Leyden WA, Manos MM, Geiger AM, Weinmann S, Mouchawar J, Bischoff K, et al. Cervical cancer in women with comprehensive health care access: attributable factors in the screening process. J Natl Cancer Inst. 2005;97(9):675–83. [PubMed] 46. Chantler C. The role and education of doctors in the delivery of health care. Lancet. 1999;353(9159):1178–81. [PubMed] 47. Ortiz E, Meyer G, Burstin H. The role of clinical informatics in the Agency for Healthcare Research and Quality’s efforts to improve patient safety. Proc AMIA Symp. 2001:508–12. [PMC free article] [PubMed] 48. Savitz L. Envisioning safer healthcare. Healthc Pap. 2004;5(3):69–74. discussion 82–4. [PubMed]
49. Pava CHP. Managing new office technology : an organizational strategy. New York. London: Free Press ; Collier Macmillan; 1983.
50. Berwick DM. A user’s manual for the IOM’s ‘Quality Chasm’ report. Health Aff (Millwood) 2002;21(3):80–90. [PubMed] 51. James BC. Making it easy to do it right. N Engl J Med. 2001;345(13):991–3. [PubMed]
52. Nielsen J. Usability engineering. Boston: Academic Press; 1993.
53. Cimino JJ, Patel VL, Kushniruk AW. Studying the human-computer-terminology interface. J Am Med Inform Assoc. 2001;8(2):163–73. [PMC free article] [PubMed] 54. Epstein RM. The science of patient-centered care. J Fam Pract. 2000;49(9):805–7. [PubMed] 55. Epstein RM, Franks P, Fiscella K, Shields CG, Meldrum SC, Kravitz RL, et al. Measuring patient-centered communication in patient-physician consultations: theoretical and practical issues. Soc Sci Med. 2005;61(7):1516–28. [PubMed] 56. Schweikhart SB. Reengineering the work of caregivers: role redefinition, team structures, and organizational redesign. Hosp Health Serv Adm. 1996;41(1):19–36. [PubMed] 57. Gustafson DH, McTavish FM, Boberg E, Owens BH, Sherbeck C, Wise M, et al. Empowering patients using computer based health support systems. Qual Health Care. 1999;8(1):49–56. [PMC free article] [PubMed]
58. Gustafson D. Engineering the Patient and Family into the Patient Care Team. In: Reid Proctor P, Compton W Dale, Grossman Jerome H, Fanjiang Gary., editors. Building a Better Delivery System: A New Engineering/Health Care Partnership. Washington DC: National Academies of Science; 2005. pp. 125–128.
59. Arora NK. Interacting with cancer patients: the significance of physicians’ communication behavior. Soc Sci Med. 2003;57(5):791–806. [PubMed] 60. Parker RM, Gazmararian JA. Health literacy: essential for health communication. J Health Commun. 2003;8 (Suppl 1):116–8. [PubMed]
61. Weed L. Opening the black box of clinical judgment-an overview. Interview by Abi Berger Bmj. 1999;319(7220):1279.
62. Norman DA, Draper SW. User centered system design : new perspectives on human-computer interaction. Hillsdale, N.J.: L. Erlbaum Associates; 1986.
63. Norman DA. 1st Basic paperback. New York: Basic Books; 2002. The design of everyday things.
64. James BC. Implementing practice guidelines through clinical quality improvement. Front Health Serv Manage. 1993;10(1):3–37. discussion 54–6. [PubMed]
65. James B. E-Health: Steps On The Road To Interoperability. Health Aff (Millwood) 2005
66. Bederson B, Shneiderman B. The craft of information visualization : readings and reflections. Amsterdam ; Boston: Morgan Kaufmann; 2003.
67. Card SK, Mackinlay JD, Shneiderman B. Readings in information visualization : using vision to think. San Francisco, Calif.: Morgan Kaufmann Publishers; 1999.
68. Plaisant C, Mushlin R, Snyder A, Li J, Heller D, Shneiderman B. LifeLines: Using Visualization to Enhance Navigation and Analysis of Patient Records. 1998 American Medical Informatic Association Annual Fall Symposium 1998; Orlando: American Medical Informatic Association; 1998.
69. Lipkus IM, Hollands JG. The visual communication of risk. J Natl Cancer Inst Monogr. 1999;(25):149–63. [PubMed] 70. Lipkus IM, Klein WM, Rimer BK. Communicating breast cancer risks to women using different formats. Cancer Epidemiol Biomarkers Prev. 2001;10(8):895–8. [PubMed] 71. Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989;81(24):1879–86. [PubMed] 72. Rimer BK, Conaway M, Lyna P, Glassman B, Yarnall KS, Lipkus I, et al. The impact of tailored interventions on a community health center population. Patient Educ Couns. 1999;37(2):125–40. [PubMed] 73. Veenstra TD, Conrads TP, Hood BL, Avellino AM, Ellenbogen RG, Morrison RS. Biomarkers: mining the biofluid proteome. Mol Cell Proteomics. 2005;4(4):409–18. [PubMed] 74. Ross JS, Schenkein DP, Pietrusko R, Rolfe M, Linette GP, Stec J, et al. Targeted therapies for cancer 2004. Am J Clin Pathol. 2004;122(4):598–609. [PubMed] 75. Kreuter MW, Wray RJ. Tailored and targeted health communication: strategies for enhancing information relevance. Am J Health Behav. 2003;27 (Suppl 3):S227–32. [PubMed] 76. Suggs LS. A 10-year retrospective of research in new technologies for health communication. J Health Commun. 2006;11(1):61–74. [PubMed] 77. Iezzoni LI. Toward universal design in assessing health care experiences. Med Care. 2002;40(9):725–8. [PubMed]
78. Hesse BW. Curb cuts in the virtual community: Telework and persons with disabilities. 28th Annual Hawaii International Conference on System Sciences; 1995; Maui, HI. 1995. pp. 418–425.
79. Chiang MF, Starren J. Evaluation of consumer health website accessibility by users with sensory and physical disabilities. Medinfo. 2004;11(Pt 2):1128–32. [PubMed] 80. Rudd RE, Comings JP, Hyde JN. Leave no one behind: improving health and risk communication through attention to literacy. J Health Commun. 2003;8 (Suppl 1):104–15. [PubMed] 81. Rudd RE, Kaphingst K, Colton T, Gregoire J, Hyde J. Rewriting public health information in plain language. J Health Commun. 2004;9(3):195–206. [PubMed]
82. Koyani S, Bailey RW, Nall JR. Research-Based Web Design and Usability Guidelines. Bethesda, MD: National Institutes of Health; 2003. Report No.: NIH Publication No. 03–5424.
83. Shneiderman B, Plaisant C. Designing the user interface : strategies for effective human-computer interaction. 4. Boston: Pearson/Addison Wesley; 2004.
84. Kaphingst KA, Zanfini CJ, Emmons KM. Accessibility of web sites containing colorectal cancer information to adults with limited literacy (United States) Cancer Causes Control. 2006;17(2):147–51. [PubMed]
85. Landauer TK. The trouble with computers : usefulness, usability, and productivity. Cambridge, Mass.: MIT Press; 1995.
86. Hesse BW, Werner CM, Altman I. Temporal aspects of computer-mediated communication. Computers in Human Behavior. 1988;4(2):147–165.
87. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12(5):505–16. [PMC free article] [PubMed]
88. Zuboff S, Maxmin J. The support economy : why corporations are failing individuals and the next episode of capitalism. New York: Viking; 2002.
89. Eng TR. Emerging technologies for cancer prevention and other population health challenges. J Med Internet Res. 2005;7(3):e30. [PMC free article] [PubMed] 90. Detmer DE. Building the national health information infrastructure for personal health, health care services, public health, and research. BMC Med Inform Decis Mak. 2003;3:1. [PMC free article] [PubMed] 91. Deering MJ. Developing the health information infrastructure in the United States. Stud Health Technol Inform. 2002;80:121–8. [PubMed] 92. Abrams DB. Applying transdisciplinary research strategies to understanding and eliminating health disparities. Health Educ Behav. 2006;33(4):515–31. [PubMed]
93. Shneiderman B. Software psychology : human factors in computer and information systems. Cambridge, Mass.: Winthrop Publishers; 1980.
94. Gladwell M. The tipping point : how little things can make a big difference. 1. Boston: Back Bay Books; 2002.