PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jamiaAlertsAuthor InstructionsSubmitAboutJAMIA - The Journal of the American Medical Informatics Association
 
J Am Med Inform Assoc. 2012 Jul-Aug; 19(4): 549–554.
Published online 2011 December 1. doi:  10.1136/amiajnl-2011-000382
PMCID: PMC3384102

Triaging patients at risk of influenza using a patient portal

Abstract

Vanderbilt University has a widely adopted patient portal, MyHealthAtVanderbilt, which provides an infrastructure to deliver information that can empower patient decision making and enhance personalized healthcare. An interdisciplinary team has developed Flu Tool, a decision-support application targeted to patients with influenza-like illness and designed to be integrated into a patient portal. Flu Tool enables patients to make informed decisions about the level of care they require and guides them to seek timely treatment as appropriate. A pilot version of Flu Tool was deployed for a 9-week period during the 2010–2011 influenza season. During this time, Flu Tool was accessed 4040 times, and 1017 individual patients seen in the institution were diagnosed as having influenza. This early experience with Flu Tool suggests that healthcare consumers are willing to use patient-targeted decision support. The design, implementation, and lessons learned from the pilot release of Flu Tool are described as guidance for institutions implementing decision support through a patient portal infrastructure.

Keywords: Medical informatics applications, medical informatics computing, personal health records, support, US gov't, user-computer interface, ethical study methods, text generation from discrete data, classical experimental and quasi-experimental study methods (lab and field), developing and refining EHR data standards (including image standards), controlled terminologies and vocabularies, portal, patient, MyHealthatVanderbilt, PHR

Objective

Healthcare organizations using electronic health record (EHR) systems are increasingly implementing patient portals to provide patients access to the healthcare system.1–6 Patient portals provide an infrastructure for patients to access and review their medical record,2 4 7–18 manage medication therapy,9 19–25 exchange electronic messages with their healthcare providers,8 15 25–34 complete medical and administrative forms,3 8 35 browse educational materials,12 36 37 and receive health and appointment reminders. Such patient portals benefit both patients and healthcare providers by increasing efficiency and productivity of care.1 9 26 Examples include the pioneering work of Masys with the PCASSO (Patient-Centered Access to Secure Systems Online) system at the University of California, San Diego,15 26 38–40 and systems at Kaiser Permanente, the Veterans Administration,4 41 New York Presbyterian Hospital,42 Vanderbilt University Medical Center (VUMC),12 43 and Partners Healthcare.2 Patient portals have been shown to enhance patient-provider communication, increase overall satisfaction with care, increase access to health information, and improve disease management for patients with chronic diseases, including diabetes.9 10 44–50

Although much has been written about patient portals in the biomedical literature, there is little research evaluating their ability to deliver patient-targeted decision support. Information technologies designed to support healthcare provider decision making, including messages suggesting specific clinical decisions or actions, have become an increasingly prevalent component of EHR systems.6 51–67 Decision support targeting patients directly may have a rate of success equal to or higher than decision support targeting healthcare providers, as supported by a number of recent investigations.68–83 Health information technologies providing patient education, reminders, online education, and targeted decision aids have been described.84–106 Linking patient-targeted decision aids to a patient portal allows the patients to receive personalized content and take actions based on the information they receive, such as sending messages directly to their healthcare provider.

We developed an influenza-screening application called the Flu Tool, designed to leverage the infrastructure from MyHealthAtVanderbilt (MHAV), a widely adopted patient portal, and to empower patients' decision making related to a specific disease syndrome requiring prompt treatment for optimal clinical outcomes. Flu Tool addresses the challenge of triaging patients with an influenza-like illness to pursue the most appropriate level of care in an efficient way. Although initiated to address concerns raised by the pH1N1 pandemic, the Flu Tool was designed to apply to all strains of seasonal influenza during a typical influenza season. The model and infrastructure supporting the Flu Tool are generalizable to other types of patient-targeted decision support. In this brief communication, we describe the implementation, pilot deployment, and lessons learned from this pilot release of a patient-targeted decision-support tool developed for a patient portal.

Materials and methods

Setting

The VUMC has a locally developed patient portal called MHAV43 that is fully integrated into the institutional EHR system. MHAV currently has over 150 000 registered users, and, on average, 5500 users access the site each day. Core functionality includes direct, bidirectional, secure electronic messaging between patients and healthcare providers (used 673 971 times in 2010), access to laboratory, imaging and vital sign testing results (702 980 times in 2010) and immunization records (37 451 times in 2010), and an ability to review billing and insurance information (250 985 times in 2010). The MHAV system has also been successfully used to deliver targeted educational materials to patients based on their clinical problems.12 The VUMC has established a series of policies, procedures and best workflow practices for patient portals.43

The Flu Tool: design phase

In creating the Flu Tool, we began with a design phase, during which we sought to specify the software components and algorithms that would best provide decision support for patients, without specific concern about potential implementation challenges. The goals were to: envision a system that could deliver appropriate, time-sensitive, and patient-initiated care to patients with an influenza-like illness; facilitate thorough clinical documentation; and operate within normal clinical workflows. A multidisciplinary team developed a series of algorithms and educational materials for guiding patients with influenza-like illness who use the MHAV system. Team members included biomedical informaticists (authors STR, SS, GPJ), experts in infectious diseases and epidemiology (TLD, TRT), hospital and clinical management (TM, RH, JJ, SM), and clinical information flow (JJ, SM), and the team had oversight from experts in law and policy, from the committee that provides oversight to MHAV development and use, and from VUMC leadership. The project team formulated an approach for extending the existing MHAV messaging infrastructure to include the algorithms for influenza screening that would both triage messages from high-risk patients to be flagged for immediate attention, and advise patients how to escalate care quickly when appropriate. The formulation process took place through an iterative procedure that involved conversations with institutional stakeholders, reviewing then-current practice standards for influenza management,107 reviewing current institutional policies related to MHAV system use, and mapping common workflow patterns around MHAV patient-provider messaging.

The Flu Tool algorithms present patients with a series of questions to determine whether their symptoms suggested a serious or complicated respiratory condition, typical influenza infection, or a minor viral respiratory illness. The Flu Tool was initially designed to accommodate the following workflow. A patient with an influenza-like illness seeking medical advice would log into the MHAV portal, which would display a link to the Flu Tool. The patient would then be prompted to answer questions about their illness and any potential comorbid conditions. Patients' answers would lead to one of four outcomes: (1) they would be advised that they may have a serious illness and should engage the medical system immediately; (2) they would be advised that they have symptoms consistent with influenza and may require antiviral therapy, and a high-priority message would be sent through MHAV to their healthcare provider; (3) they would be advised that they have symptoms consistent with influenza but that antiviral therapy is unlikely to be of benefit, and a normal-priority message would be sent through MHAV to their healthcare provider; or (4) they would be advised that they do not have symptoms consistent with influenza, and a normal-priority message would be sent through MHAV to their healthcare provider. Messages sent as a result of navigating the algorithms would include the answers the patient provided to the questionnaire. For all patients who would potentially benefit from antiviral therapy, their case would be reviewed and any treatment prescribed by the licensed healthcare provider who normally addresses the high-priority message sent through MHAV for a given clinic. In the case that the patient filled out the questionnaire outside of normal business hours and qualified for antiviral therapy, he or she would be advised to call an on-call healthcare provider directly, rather than send a message through the MHAV system and wait for a response. A sample algorithm is presented in figure 1.

Figure 1
A sample algorithm.

The Flu Tool: implementation phase

The Flu Tool was piloted coincident with the 2010–2011 influenza season in Middle Tennessee. Specifically, it was deployed during the 9-week period from February 8 through April 13, 2011. This period overlapped with the CDC assessment of influenza activity in Middle Tennessee as becoming ‘widespread’ and when Google Flu reported influenza activity in Nashville as being ‘high’; these designations are given by each system to the highest category of influenza activity. To promote usage, the Flu Tool pilot was deployed through both (1) institutional consumer-oriented websites for the VUMC and Children's Hospital and (2) a public application in Facebook. The Flu Tool was advertised through a number of channels to alert patients that it was available, including a press release to local media, electronic mail messages to the VUMC community, and in institutional newspapers.

We piloted Flu Tool by implementing just the MHAV components as described above, without the links to the patient-provider messaging system. This was primarily due to concerns that institutional clinical leaders had in launching the messaging system without around-the-clock human oversight and monitoring, such as by a 24 h/7 day nurse triage, which would have been out-of-scope for this pilot. In particular, patients might: (1) send messages outside of business hours even though the Flu Tool advised them to call their healthcare provider instead (eg, if their symptoms were consistent with influenza and antiviral medications were indicated, but it was outside normal business hours); (2) send messages outside business hours with the expectation that they would receive a call back even though the Flu Tool advised them that they did not need immediate care (eg, when they would not benefit from antiviral therapy and had no evidence of serious illness); or (3) send a message to the wrong healthcare provider (eg, to an ophthalmologist they had previously seen, who would therefore be listed as a potential recipient for patient portal messages).

Observations

During the 9 weeks that the Flu Tool was deployed, a total of 1017 patients seen at VUMC and affiliated clinics received an ICD-9 encoded diagnosis for influenza (ie, ICD-9 codes 487.xx or 488.xx). During this same period, the Flu Tool was accessed a total of 4040 times: 684 times within the MHAV system, 2928 times on the general consumer-oriented VUMC website, and 428 times on the general consumer-oriented Vanderbilt Children's Hospital website.

Discussion

Patients are taking an increasingly active role in their own healthcare delivery. This article describes an influenza management tool as a model for patient-targeted decision-support applications that can be delivered through a patient portal. The Flu Tool is designed to empower patient decision making in several ways. First, by being accessible through a patient portal, the tool's use is patient-initiated and patient-driven. Second, the questions and algorithms included in the triage system are designed to educate patients about the most likely cause of their symptoms. Third, for patients requiring escalation of care, the system design provides specific tools or guidance based on whether or not the patient's primary care services are accessible by the patient portal. For those cases where patients meet criteria for receiving treatment with an antiviral medication and can communicate with providers through a patient portal, treatment can be initiated quickly without a clinic visit.

The Flu Tool was fully deployed during the 2010–2011 influenza season, with limited integration into the MHAV patient portal. Although the Flu Tool was widely used, with its limited integration, it did not take advantage of the connections to the EHR or the integrated patient-provider messaging capabilities unique to a patient portal. We believe that the usage was sufficient to demonstrate a willingness of healthcare consumers to consult patient-targeted decision support; Flu Tool was used four times more often than the number of patients diagnosed with influenza during an inpatient or outpatient clinical encounter at VUMC. The Flu Tool was implemented during the local influenza spike, so there was not much lead-in time to advertise to patients (ie, potential users), and the fact that implementation excluded the messaging integration may have limited usage.

The deployed version did include patient-targeted decision-support algorithms and educational materials, and it provided the patient with general advice about whether they should escalate their level of care, such as whether they needed to see a healthcare provider immediately. Our usage data inform institutional clinical leaders what type of support would be necessary to release a fully integrated version of Flu Tool in MHAV. With full integration, all Flu Tool-initiated messages could be audited in real time to ensure that messages requiring immediate follow-up were addressed in a timely manner. With this service available, it may be possible to allow urgent messaging at any hour or on any day, irrespective of usual business hours. Anecdotal reports from the institutional leadership suggest that the Flu Tool and other patient-targeted decision-support tools have convinced them to consider full 24 h nurse call backup.

The model underlying the Flu Tool can serve as a template and can inform a set of workflows that would support any number of patient-targeted decision-support tools. While the contents and the specific algorithms might change, the infrastructure, policies and procedures should be constant and reusable across the institution, and may generalize to other clinical sites and institutions. By contrast, while the Flu Tool implementation was designed to respect existing workflows, institutional leadership took a relatively conservative view of potential patient risk. As a result, they requested that workflow and staffing results be changed (at some future point) to include 24 h/7 day nursing triage before integration of the Flu Tool with messaging.

As patient portals become more prevalent in settings of healthcare delivery, it will be important to measure their influence on patient-driven decision making. A more complete rollout of the Flu Tool and other portal-based patient-targeted decision-support systems will need to incorporate plans for system evaluation. Investigators have previously posited research agendas for patient portals,108 109 including evaluations of adoption, architecture, privacy and security, and their use for managing populations of patients. The limited implementation in the 2010–2011 influenza season reduced our ability to perform a formal evaluation. In the current form and during the recent 9-week implementation, the Flu Tool likely had an unmeasurable impact on healthcare utilization. With a deeper and longer implementation that includes links to the MHAV messaging system, it is possible that any impact on outcomes will be measurable. In a study evaluating this, we would likely assess a number of outcome variables, including: user demographics, patterns of algorithm recommendations, rates of MHAV users versus non-users seeking in-person care for influenza-like illness, and possibly rates of acute care and emergency department visits over several seasons. We would also hope to assess qualitative outcomes, including perceptions that patients and healthcare providers have about the Flu Tool and about patient-targeted decision support. It may also be possible to assess a series of quantitative outcomes using pre/post or time-series data obtained through queries of our institutional clinical data warehouse. This research agenda would align with those previously suggested for patient portals by other investigators.108 109

Conclusion

Patient-targeted decision-support tools can empower patients to make informed decisions about their own healthcare, to know when it is appropriate to access clinical care using patient health record messaging tools, and to identify situations in which they need to seek direct clinical care quickly. We developed a patient-targeted decision-support tool to help patients with influenza-like illness to identify situations in which they may need to escalate care, and those in which they can self-manage their illness. The Flu Tool was designed to integrate with secure electronic patient-provider messaging to guide personalized healthcare delivery. The version of the Flu Tool described in this article excluded messaging because of concerns about staffing to monitor messaging around the clock. However, without the messaging feature, the Flu Tool was heavily used. This experience highlights the importance of workflow considerations as well as staffing and stakeholder buy-in to the implementation of a novel health information technology.

Footnotes

Funding: This project was supported by a grant (to STR, number R18HS019276) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Competing interests: None.

Contributed by

Contributors: All authors have contributed substantively to this work.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Detmer D, Bloomrosen M, Raymond B, et al. Integrated personal health records: transformative tools for consumer-centric care. BMC Med Inform Decis Mak 2008;8:45. [PMC free article] [PubMed]
2. Kaelber D, Pan EC. The value of personal health record (PHR) systems. AMIA Annu Symp Proc 2008:343–7. [PMC free article] [PubMed]
3. Tang PC, Ash JS, Bates DW, et al. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006;13:121–6. [PMC free article] [PubMed]
4. Nazi KM, Woods SS. My HealtheVet PHR: a description of users and patient portal use. AMIA Annu Symp Proc 2008:1182. [PubMed]
5. Ralston JD, Hereford J, Carrell D. Use and satisfaction of a patient Web portal with a shared medical record between patients and providers. AMIA Annu Symp Proc 2006:1070. [PMC free article] [PubMed]
6. Weingart SN, Rind D, Tofias Z, et al. Who uses the patient internet portal? The PatientSite experience. J Am Med Inform Assoc 2006;13:91–5. [PMC free article] [PubMed]
7. Businger A, Buckel L, Gandhi T, et al. Patient review of selected electronic health record data improves visit experience. AMIA Annu Symp Proc 2007:887. [PubMed]
8. Haggstrom DA, Saleem JJ, Russ AL, et al. Lessons learned from usability testing of the VA's personal health record. J Am Med Inform Assoc. 2011. Oct 8. [Epub ahead of print]. [PMC free article] [PubMed]
9. Grant RW, Wald JS, Poon EG, et al. Design and implementation of a web-based patient portal linked to an ambulatory care electronic health record: patient gateway for diabetes collaborative care. Diabetes Technol Ther 2006;8:576–86. [PubMed]
10. Hess R, Bryce CL, Paone S, et al. Exploring challenges and potentials of personal health records in diabetes self-management: implementation and initial assessment. Telemed J E Health 2007;13:509–17. [PubMed]
11. Kern LM, Barron Y, Blair AJ, 3rd, et al. Electronic result viewing and quality of care in small group practices. J Gen Intern Med 2008;23:405–10. [PMC free article] [PubMed]
12. Koonce TY, Giuse DA, Beauregard JM, et al. Toward a more informed patient: bridging health care information through an interactive communication portal. J Med Libr Assoc 2007;95:77–81. [PMC free article] [PubMed]
13. Lobach DF, Willis JM, Macri JM, et al. Perceptions of Medicaid beneficiaries regarding the usefulness of accessing personal health information and services through a patient Internet portal. AMIA Annu Symp Proc 2006:509–13. [PMC free article] [PubMed]
14. Ruland CM, Brynhi H, Andersen R, et al. Developing a shared electronic health record for patients and clinicians. Stud Health Technol Inform 2008;136:57–62. [PubMed]
15. Baker DB, Masys DR. PCASSO: a design for secure communication of personal health information via the internet. Int J Med Inform 1999;54:97–104. [PubMed]
16. Ball MJ, Smith C, Bakalar RS. Personal health records: empowering consumers. J Healthc Inf Manag 2007;21:76–86. [PubMed]
17. Kahn JS, Hilton JF, Van Nunnery T, et al. Personal health records in a public hospital: experience at the HIV/AIDS clinic at San Francisco General Hospital. J Am Med Inform Assoc 2010;17:224–8. [PMC free article] [PubMed]
18. Popovich ML, Aramini JJ, Garcia M. Immunizations: the first step in a personal health record to empower patients. Stud Health Technol Inform 2008;137:286–95. [PubMed]
19. Montelius E, Astrand B, Hovstadius B, et al. Individuals appreciate having their medication record on the web: a survey of attitudes to a national pharmacy register. J Med Internet Res 2008;10:e35. [PMC free article] [PubMed]
20. Sittig DF. Personal health records on the internet: a snapshot of the pioneers at the end of the 20th Century. Int J Med Inform 2002;65:1–6. [PubMed]
21. Kim MI, Johnson KB. Personal health records: evaluation of functionality and utility. J Am Med Inform Assoc 2002;9:171–80. [PMC free article] [PubMed]
22. Schnipper JL, Gandhi TK, Wald JS, et al. Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. Inform Prim Care 2008;16:147–55. [PubMed]
23. Staroselsky M, Volk LA, Tsurikova R, et al. An effort to improve electronic health record medication list accuracy between visits: patients' and physicians' response. Int J Med Inform 2008;77:153–60. [PubMed]
24. Weingart SN, Hamrick HE, Tutkus S, et al. Medication safety messages for patients via the web portal: the MedCheck intervention. Int J Med Inform 2008;77:161–8. [PubMed]
25. Barnabei VM, O'Connor JJ, Nimphius NM, et al. The effects of a web-based tool on patient-provider communication and satisfaction with hormone therapy: a randomized evaluation. J Womens Health (Larchmt) 2008;17:147–58. [PubMed]
26. Masys D, Baker D, Butros A, et al. Giving patients access to their medical records via the internet: the PCASSO experience. J Am Med Inform Assoc 2002;9:181–91. [PMC free article] [PubMed]
27. Lin CT, Wittevrongel L, Moore L, et al. An Internet-based patient-provider communication system: randomized controlled trial. J Med Internet Res 2005;7:e47. [PMC free article] [PubMed]
28. Zhou YY, Garrido T, Chin HL, et al. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care 2007;13:418–24. [PubMed]
29. Hassol A, Walker JM, Kidder D, et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc 2004;11:505–13. [PMC free article] [PubMed]
30. Liederman EM, Lee JC, Baquero VH, et al. Patient-physician web messaging: the impact on message volume and satisfaction. J Gen Intern Med 2005;20:52–7. [PMC free article] [PubMed]
31. Hereford J, Bell E, Lee J, et al. Patient satisfaction with a Web-based patient-provider messaging system implemented across a statewide delivery system (Group Health Cooperative). AMIA Annu Symp Proc 2003:860. [PMC free article] [PubMed]
32. Ralston JD, Rutter CM, Carrell D, et al. Patient use of secure electronic messaging within a shared medical record: A cross-sectional study. J Gen Intern Med 2009;24:349–55. [PMC free article] [PubMed]
33. McGeady D, Kujala J, Ilvonen K. The impact of patient-physician web messaging on healthcare service provision. Int J Med Inform 2008;77:17–23. [PubMed]
34. Ross SE, Moore LA, Earnest MA, et al. Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004;6:e12. [PMC free article] [PubMed]
35. Kim MI, Johnson KB. Patient entry of information: evaluation of user interfaces. J Med Internet Res 2004;6:e13. [PMC free article] [PubMed]
36. Calabretta N. Consumer-driven, patient-centered health care in the age of electronic information. J Med Libr Assoc 2002;90:32–7. [PMC free article] [PubMed]
37. Tang PC, Lansky D. The missing link: bridging the patient-provider health information gap. Health Aff (Millwood) 2005;24:1290–5. [PubMed]
38. Baker DB, Masys DR, Jones RL, et al. Assurance: the power behind PCASSO security. Proc AMIA Symp 1999:666–70. [PMC free article] [PubMed]
39. Masys DR, Baker DB. Patient-Centered Access to Secure Systems Online (PCASSO): a secure approach to clinical data access via the World Wide Web. Proc AMIA Annu Fall Symp 1997:340–3. [PMC free article] [PubMed]
40. Masys DR, Baker DB, Barnhart R, et al. PCASSO: a secure architecture for access to clinical data via the Internet. Stud Health Technol Inform 1998;52:1130–4. [PubMed]
41. My HealtheVet 2009. http://www.myhealth.va.gov/(accessed 2009).
42. Cimino JJ, Patel VL, Kushniruk AW. The patient clinical information system (PatCIS): technical solutions for and experience with giving patients access to their electronic medical records. Int J Med Inform 2002;68:113–27. [PubMed]
43. Osborn CY, Rosenbloom ST, Stenner SP, et al. MyHealthAtVanderbilt: policies and procedures governing patient portal functionality. J Am Med Inform Assoc. 2011. Jul 31. [Epub ahead of print]. [PMC free article] [PubMed]
44. Nordqvist C, Hanberger L, Timpka T, et al. Health professionals' attitudes towards using a Web 2.0 portal for child and adolescent diabetes care: qualitative study. J Med Internet Res 2009;11:e12. [PMC free article] [PubMed]
45. Bryce CL, Zickmund S, Hess R, et al. Value versus user fees: perspectives of patients before and after using a web-based portal for management of diabetes. Telemed J E Health 2008;14:1035–43. [PubMed]
46. Beaudoin CE, Fernandez C, Wall JL, et al. Promoting healthy eating and physical activity short-term effects of a mass media campaign. Am J Prev Med 2007;32:217–23. [PubMed]
47. Ma C, Warren J, Phillips P, et al. Empowering patients with essential information and communication support in the context of diabetes. Int J Med Inform 2006;75:577–96. [PubMed]
48. Tang CH, Li CC, Chang GH, et al. Implementing a personalized portal combined with workflow management tools used in diabetes care. AMIA Annu Symp Proc 2003:1026. [PMC free article] [PubMed]
49. Agrawal A, Mayo-Smith MF. Adherence to computerized clinical reminders in a large healthcare delivery network. In: Fieschi M, Coiera E, Li YCJ, editors. , eds. Medinfo 2004: Proceedings of the 11th World Congress on Medical Informatics. Amsterdam: IOS Press, 2004:111–14.
50. Quinn CC, Clough SS, Minor JM, et al. WellDoc mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther 2008;10:160–8. [PubMed]
51. Anton C, Nightingale PG, Adu D, et al. Improving prescribing using a rule based prescribing system. Qual Saf Health Care 2004;13:186–90. [PMC free article] [PubMed]
52. Apkon M, Singhaviranon P. Impact of an electronic information system on physician workflow and data collection in the intensive care unit. Intensive Care Med 2001;27:122–30. [PubMed]
53. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104–12. [PMC free article] [PubMed]
54. Bates DW, Teich JM, Lee J. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6:313–21. [PMC free article] [PubMed]
55. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003;10:523–30. [PMC free article] [PubMed]
56. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280:1311–16. [PubMed]
57. Durieux P. Electronic medical alerts–so simple, so complex. N Engl J Med 2005;352:1034–6. [PubMed]
58. Adrianne C Feldstein, David H Smith, Nan R Robertson, et al. Decision Support System Design and Implementation for Outpatient Prescribing: The Safety in Prescribing Study. www.ahrq.gov/downloads/pub/advances/vol3/Feldstein.pdf. [PubMed]
59. Glassman PA, Simon B, Belperio P, et al. Improving recognition of drug interactions: benefits and barriers to using automated drug alerts. Med Care 2002;40:1161–71. [PubMed]
60. Hsieh TC, Kuperman GJ, Jaggi T, et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004;11:482–91. [PMC free article] [PubMed]
61. Kuperman GJ, Teich JM, Gandhi TK, et al. Patient safety and computerized medication ordering at Brigham and Women's Hospital. Jt Comm J Qual Improv 2001;27:509–21. [PubMed]
62. Miller RA, Waitman LR, Chen S, et al. The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt. J Biomed Inform 2005;38:469–85. [PMC free article] [PubMed]
63. Rosenbloom ST, Geissbuhler AJ, Dupont WD, et al. Effect of CPOE user interface design on user-initiated access to educational and patient information during clinical care. J Am Med Inform Assoc 2005;12:458–73. [PMC free article] [PubMed]
64. Shah NR, Seger AC, Seger DL, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006;13:5–11. [PMC free article] [PubMed]
65. Tamblyn R, Huang A, Perreault R, et al. The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ 2003;169:549–56. [PMC free article] [PubMed]
66. Teich JM, Merchia PR, Schmiz JL, et al. Effects of computerized physician order entry on prescribing practices. Arch Intern Med 2000;160:2741–7. [PubMed]
67. Denekamp Y. Clinical decision support systems for addressing information needs of physicians. Isr Med Assoc J 2007;9:771–6. [PubMed]
68. Holbrook A, Thabane L, Keshavjee K, et al. ; COMPETE II Investigators Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ 2009;181:37–44. [PMC free article] [PubMed]
69. Sequist TD, Zaslavsky AM, Marshall R, et al. Patient and physician reminders to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med 2009;169:364–71. [PMC free article] [PubMed]
70. Volandes AE, Paasche-Orlow MK, Barry MJ, et al. Video decision support tool for advance care planning in dementia: randomised controlled trial. BMJ 2009;338:b2159. [PMC free article] [PubMed]
71. Littenberg B, MacLean CD, Zygarowski K, et al. The Vermedx Diabetes Information System reduces healthcare utilization. Am J Manag Care 2009;15:166–70. [PubMed]
72. Kuppermann M, Norton ME, Gates E, et al. Computerized prenatal genetic testing decision-assisting tool: a randomized controlled trial. Obstet Gynecol 2009;113:53–63. [PubMed]
73. Harari D, Iliffe S, Kharicha K, et al. Promotion of health in older people: a randomised controlled trial of health risk appraisal in British general practice. Age Ageing 2008;37:565–71. [PubMed]
74. Ten Wolde GB, Dijkstra A, van Empelen P, et al. Long-term effectiveness of computer-generated tailored patient education on benzodiazepines: a randomized controlled trial. Addiction 2008;103:662–70. [PubMed]
75. Kypri K, Langley JD, Saunders JB, et al. Randomized controlled trial of web-based alcohol screening and brief intervention in primary care. Arch Intern Med 2008;168:530–6. [PubMed]
76. Matheny ME, Gandhi TK, Orav EJ, et al. Impact of an automated test results management system on patients' satisfaction about test result communication. Arch Intern Med 2007;167:2233–9. [PubMed]
77. Priebe S, McCabe R, Bullenkamp J, et al. Structured patient-clinician communication and 1-year outcome in community mental healthcare: cluster randomised controlled trial. Br J Psychiatry 2007;191:420–6. [PubMed]
78. Thomson RG, Eccles MP, Steen IN, et al. A patient decision aid to support shared decision-making on anti-thrombotic treatment of patients with atrial fibrillation: randomised controlled trial. Qual Saf Health Care 2007;16:216–23. [PMC free article] [PubMed]
79. Saitz R, Palfai TP, Freedner N, et al. Screening and brief intervention online for college students: the ihealth study. Alcohol Alcohol 2007;42:28–36. [PubMed]
80. Cintron A, Phillips R, Hamel MB. The effect of a web-based, patient-directed intervention on knowledge, discussion, and completion of a health care proxy. J Palliat Med 2006;9:1320–8. [PubMed]
81. Strecher VJ, Shiffman S, West R. Moderators and mediators of a web-based computer-tailored smoking cessation program among nicotine patch users. Nicotine Tob Res 2006;8(Suppl 1):S95–101. [PubMed]
82. Japuntich SJ, Zehner ME, Smith SS, et al. Smoking cessation via the internet: a randomized clinical trial of an internet intervention as adjuvant treatment in a smoking cessation intervention. Nicotine Tob Res 2006;8(Suppl 1):S59–67. [PubMed]
83. Poon EG, Wald J, Schnipper JL, et al. Empowering patients to improve the quality of their care: design and implementation of a shared health maintenance module in a US integrated healthcare delivery network. Stud Health Technol Inform 2007;129:1002–6. [PubMed]
84. Bock BC, Graham AL, Whiteley JA, et al. A review of web-assisted tobacco interventions (WATIs). J Med Internet Res 2008;10:e39. [PMC free article] [PubMed]
85. Buis LR, Poulton TA, Holleman RG, et al. Evaluating Active U: an Internet-mediated physical activity program. BMC Public Health 2009;9:331. [PMC free article] [PubMed]
86. Buller DB, Woodall WG, Zimmerman DE, et al. Randomized trial on the 5 a day, the Rio Grande Way Website, a web-based program to improve fruit and vegetable consumption in rural communities. J Health Commun 2008;13:230–49. [PubMed]
87. Buller MK, Kane IL, Martin RC, et al. Randomized trial evaluating computer-based sun safety education for children in elementary school. J Cancer Educ 2008;23:74–9. [PubMed]
88. Carpenter KM, Stoner SA, Mikko AN, et al. Efficacy of a web-based intervention to reduce sexual risk in men who have sex with men. AIDS Behav 2009;14:549–57. [PMC free article] [PubMed]
89. Chaudhry R, Scheitel SM, McMurtry EK, et al. Web-based proactive system to improve breast cancer screening: a randomized controlled trial. Arch Intern Med 2007;167:606–11. [PubMed]
90. Dumrongpakapakorn P, Hopkins K, Sherwood P, et al. Computer-mediated patient education: opportunities and challenges for supporting women with ovarian cancer. Nurs Clin North Am 2009;44:339–54. [PMC free article] [PubMed]
91. Dunton GF, Robertson TP. A tailored Internet-plus-email intervention for increasing physical activity among ethnically-diverse women. Prev Med 2008;47:605–11. [PubMed]
92. Fry JP, Neff RA. Periodic prompts and reminders in health promotion and health behavior interventions: systematic review. J Med Internet Res 2009;11:e16. [PMC free article] [PubMed]
93. Hartmann CW, Sciamanna CN, Blanch DC, et al. A website to improve asthma care by suggesting patient questions for physicians: qualitative analysis of user experiences. J Med Internet Res 2007;9:e3. [PMC free article] [PubMed]
94. Huang SJ, Hung WC, Chang M, et al. The effect of an internet-based, stage-matched message intervention on young Taiwanese women's physical activity. J Health Commun 2009;14:210–27. [PubMed]
95. Huang YW, Hung CH. The effect of health education through the internet on university female students' hepatitis B knowledge and cognition. J Clin Nurs 2009;18:3342–8. [PubMed]
96. Liebreich T, Plotnikoff RC, Courneya KS, et al. Diabetes NetPLAY: A physical activity website and linked email counselling randomized intervention for individuals with type 2 diabetes. Int J Behav Nutr Phys Act 2009;6:18. [PMC free article] [PubMed]
97. Lustria ML, Cortese J, Noar SM, et al. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns 2009;74:156–73. [PubMed]
98. Neve M, Morgan PJ, Jones PR, et al. Effectiveness of web-based interventions in achieving weight loss and weight loss maintenance in overweight and obese adults: a systematic review with meta-analysis. Obes Rev 2009;11:306–21. [PubMed]
99. Saul JE, Schillo BA, Evered S, et al. Impact of a statewide Internet-based tobacco cessation intervention. J Med Internet Res 2007;9:e28. [PMC free article] [PubMed]
100. Saver BG, Gustafson D, Taylor TR, et al. A tale of two studies: the importance of setting, subjects and context in two randomized, controlled trials of a web-based decision support for perimenopausal and postmenopausal health decisions. Patient Educ Couns 2007;66:211–22. [PubMed]
101. Vandelanotte C, Spathonis KM, Eakin EG, et al. Website-delivered physical activity interventions a review of the literature. Am J Prev Med 2007;33:54–64. [PubMed]
102. Wallace C, Leask J, Trevena LJ. Effects of a web based decision aid on parental attitudes to MMR vaccination: a before and after study. BMJ 2006;332:146–9. [PMC free article] [PubMed]
103. Woodall WG, Buller DB, Saba L, et al. Effect of emailed messages on return use of a nutrition education website and subsequent changes in dietary behavior. J Med Internet Res 2007;9:e27. [PMC free article] [PubMed]
104. Jackson CL, Bolen S, Brancati FL, et al. A systematic review of interactive computer-assisted technology in diabetes care. Interactive information technology in diabetes care. J Gen Intern Med 2006;21:105–10. [PMC free article] [PubMed]
105. Bussey-Smith KL, Rossen RD. A systematic review of randomized control trials evaluating the effectiveness of interactive computerized asthma patient education programs. Ann Allergy Asthma Immunol 2007;98:507–16; quiz 16, 566. [PubMed]
106. Walters ST, Wright JA, Shegog R. A review of computer and Internet-based interventions for smoking behavior. Addict Behav 2006;31:264–77. [PubMed]
107. Updated Interim Recommendations for the Use of Antiviral Medications in the Treatment and Prevention of Influenza for the 2009–2010 Season. Atlanta: Centers for Disease Control and Prevention, 2009.
108. Kaelber DC, Jha AK, Johnston D, et al. A research agenda for personal health records (PHRs). J Am Med Inform Assoc 2008;15:729–36. [PMC free article] [PubMed]
109. Nazi KM, Hogan TP, Wagner TH, et al. Embracing a health services research perspective on personal health records: lessons learned from the VA My HealtheVet system. J Gen Intern Med 2009;25(Suppl 1):62–7. [PMC free article] [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association