PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (102)
 

Clipboard (0)
None

Select a Filter Below

Year of Publication
more »
1.  Rights and responsibilities of users of electronic health records 
doi:10.1503/cmaj.111599
PMCID: PMC3447016  PMID: 22331971
2.  A survey of informatics platforms that enable distributed comparative effectiveness research using multi-institutional heterogeneous clinical data 
Medical care  2012;50(Suppl):S49-S59.
Comparative Effectiveness Research (CER) has the potential to transform the current healthcare delivery system by identifying the most effective medical and surgical treatments, diagnostic tests, disease prevention methods and ways to deliver care for specific clinical conditions. To be successful, such research requires the identification, capture, aggregation, integration, and analysis of disparate data sources held by different institutions with diverse representations of the relevant clinical events. In an effort to address these diverse demands, there have been multiple new designs and implementations of informatics platforms that provide access to electronic clinical data and the governance infrastructure required for inter-institutional CER. The goal of this manuscript is to help investigators understand why these informatics platforms are required and to compare and contrast six, large-scale, recently funded, CER-focused informatics platform development efforts. We utilized an 8-dimension, socio-technical model of health information technology use to help guide our work. We identified six generic steps that are necessary in any distributed, multi-institutional CER project: data identification, extraction, modeling, aggregation, analysis, and dissemination. We expect that over the next several years these projects will provide answers to many important, and heretofore unanswerable, clinical research questions.
doi:10.1097/MLR.0b013e318259c02b
PMCID: PMC3415281  PMID: 22692259
Methods; Comparative Effectiveness Research; Organization and Administration; Medical Informatics; Methods
3.  Defining Health Information Technology-related Errors: New Developments Since To Err is Human* 
Archives of internal medicine  2011;171(14):1281-1284.
Summary
Despite its promise, recent literature has revealed possible safety hazards of health information technology (HIT) use. The Office of the National Coordinator for HIT recently sponsored an Institute of Medicine committee to synthesize evidence and experience from the field on how HIT affects patient safety. To lay the groundwork for defining, measuring, and analyzing HIT-related safety hazards, we propose that Health information technology-related error occurs anytime HIT is unavailable for use, malfunctions during use, is used incorrectly by someone, or when HIT interacts with another system component incorrectly, resulting in data being lost or incorrectly entered, displayed, or transmitted. These errors, or the decisions that result from them, significantly increase the risk of adverse events and patient harm. In this paper, we describe how a socio-technical approach can be used to understand the complex origins of HIT errors, which may have roots in rapidly evolving technological, professional, organizational, and policy initiatives.
doi:10.1001/archinternmed.2011.327
PMCID: PMC3677061  PMID: 21788544
Electronic Health Records; Health Information Technology; Patient Safety; Errors
4.  Clinical Decision Support Alert Appropriateness: A Review and Proposal for Improvement 
The Ochsner Journal  2014;14(2):195-202.
Background
Many healthcare providers are adopting clinical decision support (CDS) systems to improve patient safety and meet meaningful use requirements. Computerized alerts that prompt clinicians about drug-allergy, drug-drug, and drug-disease warnings or provide dosing guidance are most commonly implemented. Alert overrides, which occur when clinicians do not follow the guidance presented by the alert, can hinder improved patient outcomes.
Methods
We present a review of CDS alerts and describe a proposal to develop novel methods for evaluating and improving CDS alerts that builds upon traditional informatics approaches. Our proposal incorporates previously described models for predicting alert overrides that utilize retrospective chart review to determine which alerts are clinically relevant and which overrides are justifiable.
Results
Despite increasing implementations of CDS alerts, detailed evaluations rarely occur because of the extensive labor involved in manual chart reviews to determine alert and response appropriateness. Further, most studies have solely evaluated alert overrides that are appropriate or justifiable. Our proposal expands the use of web-based monitoring tools with an interactive dashboard for evaluating CDS alert and response appropriateness that incorporates the predictive models. The dashboard provides 2 views, an alert detail view and a patient detail view, to provide a full history of alerts and help put the patient's events in context.
Conclusion
The proposed research introduces several innovations to address the challenges and gaps in alert evaluations. This research can transform alert evaluation processes across healthcare settings, leading to improved CDS, reduced alert fatigue, and increased patient safety.
PMCID: PMC4052586  PMID: 24940129
Decision support systems–clinical; electronic health records; medical order entry systems; medication errors; prevention and control; reminder systems
5.  The state of the art in clinical knowledge management: An inventory of tools and techniques 
Purpose
To explore the need for, and use of, high-quality, collaborative, clinical knowledge management (CKM) tools and techniques to manage clinical decision support content.
Methods
In order to better understand the current state of the art in CKM, we developed a survey of potential CKM tools and techniques. We conducted an exploratory study by querying a convenience sample of respondents about their use of specific practices in CKM.
Results
The following tools and techniques should be priorities in organizations interested in developing successful computer-based provider order entry (CPOE) and clinical decision support (CDS) implementations: 1) A multidisciplinary team responsible for creating and maintaining the clinical content; 2) An external organizational repository of clinical content with web-based viewer that allows anyone in the organization to review it; 3) An online, collaborative, interactive, internet-based tool to facilitate content development; 4) An enterprise-wide tool to maintain the controlled clinical terminology concepts. Even organizations that have been successfully using Computer-based Provider Order Entry with advanced Clinical Decision Support features for well over 15 years are not using all of the CKM tools or practices that we identified.
Conclusions
If we are to further stimulate progress in the area of clinical decision support, we must continue to develop and refine our understanding and use of advanced CKM capabilities.
doi:10.1016/j.ijmedinf.2009.09.003
PMCID: PMC2895508  PMID: 19828364
7.  Information Overload and Missed Test Results in EHR-based Settings 
JAMA internal medicine  2013;173(8):10.1001/2013.jamainternmed.61.
doi:10.1001/2013.jamainternmed.61
PMCID: PMC3822526  PMID: 23460235
diagnostic errors; patient safety; test results; care delays; health information technology; primary care; electronic health records; usability
8.  Legal, Ethical, and Financial Dilemmas in Electronic Health Record Adoption and Use 
Pediatrics  2011;127(4):e1042-e1047.
Electronic health records (EHRs) facilitate several innovations capable of reforming health care. Despite their promise, many currently unanswered legal, ethical, and financial questions threaten the widespread adoption and use of EHRs. Key legal dilemmas that must be addressed in the near-term pertain to the extent of clinicians' responsibilities for reviewing the entire computer-accessible clinical synopsis from multiple clinicians and institutions, the liabilities posed by overriding clinical decision support warnings and alerts, and mechanisms for clinicians to publically report potential EHR safety issues. Ethical dilemmas that need additional discussion relate to opt-out provisions that exclude patients from electronic record storage, sale of deidentified patient data by EHR vendors, adolescent control of access to their data, and use of electronic data repositories to redesign the nation's health care delivery and payment mechanisms on the basis of statistical analyses. Finally, one overwhelming financial question is who should pay for EHR implementation because most users and current owners of these systems will not receive the majority of benefits. The authors recommend that key stakeholders begin discussing these issues in a national forum. These actions can help identify and prioritize solutions to the key legal, ethical, and financial dilemmas discussed, so that widespread, safe, effective, interoperable EHRs can help transform health care.
doi:10.1542/peds.2010-2184
PMCID: PMC3065078  PMID: 21422090
electronic health records; ethics; medical; confidentiality
9.  Health Care Transformation Through Collaboration on Open-Source Informatics Projects: Integrating a Medical Applications Platform, Research Data Repository, and Patient Summarization 
Background
The Strategic Health IT Advanced Research Projects (SHARP) program seeks to conquer well-understood challenges in medical informatics through breakthrough research. Two SHARP centers have found alignment in their methodological needs: (1) members of the National Center for Cognitive Informatics and Decision-making (NCCD) have developed knowledge bases to support problem-oriented summarizations of patient data, and (2) Substitutable Medical Apps, Reusable Technologies (SMART), which is a platform for reusable medical apps that can run on participating platforms connected to various electronic health records (EHR). Combining the work of these two centers will ensure wide dissemination of new methods for synthesized views of patient data. Informatics for Integrating Biology and the Bedside (i2b2) is an NIH-funded clinical research data repository platform in use at over 100 sites worldwide. By also working with a co-occurring initiative to SMART-enabling i2b2, we can confidently write one app that can be used extremely broadly.
Objective
Our goal was to facilitate development of intuitive, problem-oriented views of the patient record using NCCD knowledge bases that would run in any EHR. To do this, we developed a collaboration between the two SHARPs and an NIH center, i2b2.
Methods
First, we implemented collaborative tools to connect researchers at three institutions. Next, we developed a patient summarization app using the SMART platform and a previously validated NCCD problem-medication linkage knowledge base derived from the National Drug File-Reference Terminology (NDF-RT). Finally, to SMART-enable i2b2, we implemented two new Web service “cells” that expose the SMART application programming interface (API), and we made changes to the Web interface of i2b2 to host a “carousel” of SMART apps.
Results
We deployed our SMART-based, NDF-RT-derived patient summarization app in this SMART-i2b2 container. It displays a problem-oriented view of medications and presents a line-graph display of laboratory results.
Conclusions
This summarization app can be run in any EHR environment that either supports SMART or runs SMART-enabled i2b2. This i2b2 “clinical bridge” demonstrates a pathway for reusable app development that does not require EHR vendors to immediately adopt the SMART API. Apps can be developed in SMART and run by clinicians in the i2b2 repository, reusing clinical data extracted from EHRs. This may encourage the adoption of SMART by supporting SMART app development until EHRs adopt the platform. It also allows a new variety of clinical SMART apps, fueled by the broad aggregation of data types available in research repositories. The app (including its knowledge base) and SMART-i2b2 are open-source and freely available for download.
doi:10.2196/ijmr.2454
PMCID: PMC3668611  PMID: 23722634
clinical information systems; medical informatics; knowledge bases; user-computer interface; data display; diffusion of innovation
10.  A New Socio-technical Model for Studying Health Information Technology in Complex Adaptive Healthcare Systems 
Quality & safety in health care  2010;19(Suppl 3):i68-i74.
Conceptual models have been developed to address challenges inherent in studying health information technology (HIT). This manuscript introduces an 8-dimensional model specifically designed to address the socio-technical challenges involved in design, development, implementation, use, and evaluation of HIT within complex adaptive healthcare systems. The 8 dimensions are not independent, sequential, or hierarchical, but rather are interdependent and interrelated concepts similar to compositions of other complex adaptive systems. Hardware and software computing infrastructure refers to equipment and software used to power, support, and operate clinical applications and devices. Clinical content refers to textual or numeric data and images that constitute the “language” of clinical applications. The human computer interface includes all aspects of the computer that users can see, touch, or hear as they interact with it. People refers to everyone who interacts in some way with the system, from developer to end-user, including potential patient-users. Workflow and communication are the processes or steps involved in assuring that patient care tasks are carried out effectively. Two additional dimensions of the model are internal organizational features (e.g., policies, procedures, and culture) and external rules and regulations, both of which may facilitate or constrain many aspects of the preceding dimensions. The final dimension is measurement and monitoring, which refers to the process of measuring and evaluating both intended and unintended consequences of HIT implementation and use. We illustrate how our model has been successfully applied in real-world complex adaptive settings to understand and improve HIT applications at various stages of development and implementation.
doi:10.1136/qshc.2010.042085
PMCID: PMC3120130  PMID: 20959322
11.  Clinical Decision Support for Colon and Rectal Surgery: An Overview 
Clinical decision support (CDS) has been shown to improve clinical processes, promote patient safety, and reduce costs in healthcare settings, and it is now a requirement for clinicians as part of the Meaningful Use Regulation. However, most evidence for CDS has been evaluated primarily in internal medicine care settings, and colon and rectal surgery (CRS) has unique needs with CDS that are not frequently described in the literature. The authors reviewed published literature in informatics and medical journals, combined with expert opinion to define CDS, describe the evidence for CDS, outline the implementation process for CDS, and present applications of CDS in CRS.CDS functionalities such as order sets, documentation templates, and order facilitation aids are most often described in the literature and most likely to be beneficial in CRS. Further research is necessary to identify and better evaluate additional CDS systems in the setting of CRS.
doi:10.1055/s-0033-1333644
PMCID: PMC3699146  PMID: 24436644
clinical decision support systems; colorectal surgery; electronic health records; patient safety
12.  Review of Reported Clinical Information System Adverse Events in US Food and Drug Administration Databases 
Applied clinical informatics  2011;2(1):63-74.
Summary
Background
The US FDA has been collecting information on medical devices involved in significant adverse advents since 1984. These reports have been used by researchers to advise clinicians on potential risks and complications of using these devices.
Objective
Research adverse events related to the use of Clinical Information Systems (CIS) as reported in FDA databases.
Methods
Three large, national, adverse event medical device databases were examined for reports pertaining to CIS.
Results
One hundred and twenty unique reports (from over 1.4 million reports) were found, representing 32 manufacturers. The manifestations of these adverse events included: missing or incorrect data, data displayed for the wrong patient, chaos during system downtime and system unavailable for use. Analysis of these reports illustrated events associated with system design, implementation, use, and support.
Conclusion
The identified causes can be used by manufacturers to improve their products and by clinical facilities and providers to adjust their workflow and implementation processes appropriately. The small number of reports found indicates a need to raise awareness regarding publicly available tools for documenting problems with CIS and for additional reporting and dialog between manufacturers, organizations, and users.
doi:10.4338/ACI-2010-11-RA-0064
PMCID: PMC3175794  PMID: 21938265
Electronic Health Records; Information Systems; Mandatory Reporting; Medical Errors; United States Food and Drug Administration
14.  Assessing the Anticipated Consequences of Computer-based Provider Order Entry at Three Community Hospitals Using an Open-ended, Semi-structured Survey Instrument 
Objective
To determine what “average” clinicians in organizations that were about to implement Computer-based Provider Order Entry (CPOE) were expecting to occur, we conducted an open-ended, semi-structured survey at three community hospitals.
Methods
We created an open-ended, semi-structured, interview survey template that we customized for each organization. This interview-based survey was designed to be administered orally to clinicians and take approximately five minutes to complete, although clinicians were allowed to discuss as many advantages or disadvantages of the impending system roll-out as they wanted to.
Results
Our survey findings did not reveal any overly negative, critical, problematic, or striking sets of concerns. However, from the standpoint of unintended consequences, we found that clinicians were anticipating only a few of the events, emotions, and process changes that are likely to result from CPOE.
Conclusions
The results of such an open-ended survey may prove useful in helping CPOE leaders to understand user perceptions and predictions about CPOE, because it can expose issues about which more communication, or discussion, is needed. Using the survey, implementation strategies and management techniques outlined in this paper, any chief information officer (CIO) or chief medical information officer (CMIO) should be able to adequately assess their organization's CPOE readiness, make the necessary mid-course corrections, and be prepared to deal with the currently identified unintended consequences of CPOE should they occur.
doi:10.1016/j.ijmedinf.2007.08.005
PMCID: PMC2668516  PMID: 17931963
Medical Order Entry Systems; Ethnology; Hospitals, Community; Medical Informatics
15.  Grand Challenges in Clinical Decision Support v10 
Journal of biomedical informatics  2007;41(2):387-392.
There is a pressing need for high-quality, effective means of designing, developing, presenting, implementing, evaluating, and maintaining all types of clinical decision support capabilities for clinicians, patients and consumers. Using an iterative, consensus-building process we identified a rank-ordered list of the top 10 grand challenges in clinical decision support. This list was created to educate and inspire researchers, developers, funders, and policy-makers. The list of challenges in order of importance that they be solved if patients and organizations are to begin realizing the fullest benefits possible of these systems consists of: Improve the human-computer interface; Disseminate best practices in CDS design, development, and implementation; Summarize patient-level information; Prioritize and filter recommendations to the user; Create an architecture for sharing executable CDS modules and services; Combine recommendations for patients with co-morbidities; Prioritize CDS content development and implementation; Create internet-accessible clinical decision support repositories; Use freetext information to drive clinical decision support; Mine large clinical databases to create new CDS. Identification of solutions to these challenges is critical if clinical decision support is to achieve its potential and improve the quality, safety and efficiency of healthcare.
doi:10.1016/j.jbi.2007.09.003
PMCID: PMC2660274  PMID: 18029232
16.  Improving the Effectiveness of Electronic Health Record-Based Referral Processes 
Electronic health records are increasingly being used to facilitate referral communication in the outpatient setting. However, despite support by technology, referral communication between primary care providers and specialists is often unsatisfactory and is unable to eliminate care delays. This may be in part due to lack of attention to how information and communication technology fits within the social environment of health care. Making electronic referral communication effective requires a multifaceted “socio-technical” approach. Using an 8-dimensional socio-technical model for health information technology as a framework, we describe ten recommendations that represent good clinical practices to design, develop, implement, improve, and monitor electronic referral communication in the outpatient setting. These recommendations were developed on the basis of our previous work, current literature, sound clinical practice, and a systems-based approach to understanding and implementing health information technology solutions. Recommendations are relevant to system designers, practicing clinicians, and other stakeholders considering use of electronic health records to support referral communication.
doi:10.1186/1472-6947-12-107
PMCID: PMC3492108  PMID: 22973874
17.  Standard practices for computerized clinical decision support in community hospitals: a national survey 
Objective
Computerized provider order entry (CPOE) with clinical decision support (CDS) can help hospitals improve care. Little is known about what CDS is presently in use and how it is managed, however, especially in community hospitals. This study sought to address this knowledge gap by identifying standard practices related to CDS in US community hospitals with mature CPOE systems.
Materials and Methods
Representatives of 34 community hospitals, each of which had over 5 years experience with CPOE, were interviewed to identify standard practices related to CDS. Data were analyzed with a mix of descriptive statistics and qualitative approaches to the identification of patterns, themes and trends.
Results
This broad sample of community hospitals had robust levels of CDS despite their small size and the independent nature of many of their physician staff members. The hospitals uniformly used medication alerts and order sets, had sophisticated governance procedures for CDS, and employed staff to customize CDS.
Discussion
The level of customization needed for most CDS before implementation was greater than expected. Customization requires skilled individuals who represent an emerging manpower need at this type of hospital.
Conclusion
These results bode well for robust diffusion of CDS to similar hospitals in the process of adopting CDS and suggest that national policies to promote CDS use may be successful.
doi:10.1136/amiajnl-2011-000705
PMCID: PMC3486730  PMID: 22707744
Clinical; decision support systems; medical order entry systems
18.  Use of Order Sets in Inpatient Computerized Provider Order Entry Systems: A Comparative Analysis of Usage Patterns at Seven Sites 
Background
Many computerized provider order entry (CPOE) systems include the ability to create electronic order sets: collections of clinically-related orders grouped by purpose. Order sets promise to make CPOE systems more efficient, improve care quality and increase adherence to evidence-based guidelines. However, the development and implementation of order sets can be expensive and time-consuming and limited literature exists about their utilization.
Methods
Based on analysis of order set usage logs from a diverse purposive sample of seven sites with commercially- and internally-developed inpatient CPOE systems, we developed an original order set classification system. Order sets were categorized across seven non-mutually exclusive axes: admission/discharge/transfer (ADT), perioperative, condition-specific, task-specific, service-specific, convenience, and personal. In addition, 731 unique subtypes were identified within five axes: four in ADT (S=4), three in perioperative, 144 in condition-specific, 513 in task-specific, and 67 in service-specific.
Results
Order sets (n=1,914) were used a total of 676,142 times at the participating sites during a one-year period. ADT and perioperative order sets accounted for 27.6% and 24.2% of usage respectively. Peripartum/labor, chest pain/Acute Coronary Syndrome/Myocardial Infarction and diabetes order sets accounted for 51.6% of condition-specific usage. Insulin, angiography/angioplasty and arthroplasty order sets accounted for 19.4% of task-specific usage. Emergency/trauma, Obstetrics/Gynecology/Labor Delivery and anesthesia accounted for 32.4% of service-specific usage. Overall, the top 20% of order sets accounted for 90.1% of all usage. Additional salient patterns are identified and described.
Conclusion
We observed recurrent patterns in order set usage across multiple sites as well as meaningful variations between sites. Vendors and institutional developers should identify high-value order set types through concrete data analysis in order to optimize the resources devoted to development and implementation.
doi:10.1016/j.ijmedinf.2012.04.003
PMCID: PMC3466359  PMID: 22819199
order sets; electronic health records; clinical decision support; computerized physician order entry system
19.  Emotional Aspects of Computer-based Provider Order Entry: A Qualitative Study 
Objectives: Computer-based provider order entry (CPOE) systems are implemented to increase both efficiency and accuracy in health care, but these systems often cause a myriad of emotions to arise. This qualitative research investigates the emotions surrounding CPOE implementation and use.
Methods: We performed a secondary analysis of several previously collected qualitative data sets from interviews and observations of over 50 individuals. Three researchers worked in parallel to identify themes that expressed emotional responses to CPOE. We then reviewed and classified these quotes using a validated hierarchical taxonomy of semantically homogeneous terms associated with specific emotions.
Results: The implementation and use of CPOE systems provoked examples of positive, negative, and neutral emotions. Negative emotional responses were the most prevalent, by far, in all the observations.
Conclusion: Designing and implementing CPOE systems is difficult. These systems and the implementation process itself often inspire intense emotions. If designers and implementers fail to recognize that various CPOE features and implementation strategies can increase clinicians' negative emotions, then the systems may fail to become a routine part of the clinical care delivery process. We might alleviate some of these problems by designing positive feedback mechanisms for both the systems and the organizations.
doi:10.1197/jamia.M1711
PMCID: PMC1205605  PMID: 15905478
20.  Effectiveness of an Electronic Health Record-Based Intervention to Improve Follow-up of Abnormal Pathology Results: a Retrospective Record Analysis 
Medical care  2012;50(10):898-904.
Background and Objective
On March 11, 2009, the Veterans Health Administration (VA) implemented an electronic health record (EHR)-based intervention that required all pathology results to be transmitted to ordering providers via mandatory automated notifications. We examined the impact of this intervention on improving follow-up of abnormal outpatient pathology results.
Research Design and Subjects
We extracted pathology reports from the EHR of two VA sites. From 16,738 pre- and 17,305 post-intervention reports between 09/01/2008 and 09/30/2009, we randomly selected about 5% and evaluated follow-up outcomes using a standardized chart review instrument. Documented responses to the alerted report (e.g., ordering follow-up tests or referrals, notifying patients, and prescribing/changing treatment) were recorded.
Measures
Primary outcome measures included proportion of timely follow-up responses (within 30 days) and median time to direct response for abnormal reports.
Results
Of 816 pre- and 798 post-intervention reports reviewed, 666 (81.6%) and 688 (86.2%) were abnormal. Overall, there was no apparent intervention effect on timely follow-up (69% vs. 67.1%;p=0.4) or median time to direct response (8 days vs. 8 days; p=0.7). However, logistic regression uncovered a significant intervention effect (pre-intervention OR, 0.7; 95%CI 0.5-1.0) after accounting for site-specific differences in follow-up, with a lower likelihood of timely follow-up at one site (OR,0.4; 95%CI 0.2-0.7).
Conclusion
An electronic intervention to improve test result follow-up at two VA institutions using the same EHR was found effective only after accounting for certain local contextual factors. Aggregating the effect of EHR interventions across different institutions and EHRs without controlling for contextual factors might underestimate their potential benefits.
doi:10.1097/MLR.0b013e31825f6619
PMCID: PMC3444625  PMID: 22929995
Anatomic pathology; electronic health record; communication; follow-up; post-analytic phase
21.  Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study 
Background
The purpose of this study was to identify recommended practices for computerized clinical decision support (CDS) development and implementation and for knowledge management (KM) processes in ambulatory clinics and community hospitals using commercial or locally developed systems in the U.S.
Methods
Guided by the Multiple Perspectives Framework, the authors conducted ethnographic field studies at two community hospitals and five ambulatory clinic organizations across the U.S. Using a Rapid Assessment Process, a multidisciplinary research team: gathered preliminary assessment data; conducted on-site interviews, observations, and field surveys; analyzed data using both template and grounded methods; and developed universal themes. A panel of experts produced recommended practices.
Results
The team identified ten themes related to CDS and KM. These include: 1) workflow; 2) knowledge management; 3) data as a foundation for CDS; 4) user computer interaction; 5) measurement and metrics; 6) governance; 7) translation for collaboration; 8) the meaning of CDS; 9) roles of special, essential people; and 10) communication, training, and support. Experts developed recommendations about each theme. The original Multiple Perspectives framework was modified to make explicit a new theoretical construct, that of Translational Interaction.
Conclusions
These ten themes represent areas that need attention if a clinic or community hospital plans to implement and successfully utilize CDS. In addition, they have implications for workforce education, research, and national-level policy development. The Translational Interaction construct could guide future applied informatics research endeavors.
doi:10.1186/1472-6947-12-6
PMCID: PMC3334687  PMID: 22333210
22.  A qualitative study of the activities performed by people involved in clinical decision support: recommended practices for success 
Objective
To describe the activities performed by people involved in clinical decision support (CDS) at leading sites.
Materials and methods
We conducted ethnographic observations at seven diverse sites with a history of excellence in CDS using the Rapid Assessment Process and analyzed the data using a series of card sorts, informed by Linstone's Multiple Perspectives Model.
Results
We identified 18 activities and grouped them into four areas. Area 1: Fostering relationships across the organization, with activities (a) training and support, (b) visibility/presence on the floor, (c) liaising between people, (d) administration and leadership, (e) project management, (f) cheerleading/buy-in/sponsorship, (g) preparing for CDS implementation. Area 2: Assembling the system with activities (a) providing technical support, (b) CDS content development, (c) purchasing products from vendors (d) knowledge management, (e) system integration. Area 3: Using CDS to achieve the organization's goals with activities (a) reporting, (b) requirements-gathering/specifications, (c) monitoring CDS, (d) linking CDS to goals, (e) managing data. Area 4: Participation in external policy and standards activities (this area consists of only a single activity). We also identified a set of recommendations associated with these 18 activities.
Discussion
All 18 activities we identified were performed at all sites, although the way they were organized into roles differed substantially. We consider these activities critical to the success of a CDS program.
Conclusions
A series of activities are performed by sites strong in CDS, and sites adopting CDS should ensure they incorporate these activities into their efforts.
doi:10.1136/amiajnl-2013-001771
PMCID: PMC3994853  PMID: 23999670
clinical decision support; knowledge management; governance; implementation; rapid assessment process
23.  Development and evaluation of a crowdsourcing methodology for knowledge base construction: identifying relationships between clinical problems and medications 
Objective
We describe a novel, crowdsourcing method for generating a knowledge base of problem–medication pairs that takes advantage of manually asserted links between medications and problems.
Methods
Through iterative review, we developed metrics to estimate the appropriateness of manually entered problem–medication links for inclusion in a knowledge base that can be used to infer previously unasserted links between problems and medications.
Results
Clinicians manually linked 231 223 medications (55.30% of prescribed medications) to problems within the electronic health record, generating 41 203 distinct problem–medication pairs, although not all were accurate. We developed methods to evaluate the accuracy of the pairs, and after limiting the pairs to those meeting an estimated 95% appropriateness threshold, 11 166 pairs remained. The pairs in the knowledge base accounted for 183 127 total links asserted (76.47% of all links). Retrospective application of the knowledge base linked 68 316 medications not previously linked by a clinician to an indicated problem (36.53% of unlinked medications). Expert review of the combined knowledge base, including inferred and manually linked problem–medication pairs, found a sensitivity of 65.8% and a specificity of 97.9%.
Conclusion
Crowdsourcing is an effective, inexpensive method for generating a knowledge base of problem–medication pairs that is automatically mapped to local terminologies, up-to-date, and reflective of local prescribing practices and trends.
doi:10.1136/amiajnl-2012-000852
PMCID: PMC3422843  PMID: 22582202
Electronic health records; decision support systems; clinical; knowledge bases; medication systems; hospital; data collection; clinical decision support; error management and prevention; evaluation; monitoring and surveillance; ADEs; developing/using computerized provider order entry; knowledge representations; classical experimental and quasi-experimental study methods (lab and field); designing usable (responsive) resources and systems; statistical analysis of large datasets; electronic health records; clinical summarization; user interface; patient preferences; patient-centered; heart failure; psychological; nursing; clinical information systems
24.  Towards successful coordination of electronic health record based-referrals: a qualitative analysis 
Background
Successful subspecialty referrals require considerable coordination and interactive communication among the primary care provider (PCP), the subspecialist, and the patient, which may be challenging in the outpatient setting. Even when referrals are facilitated by electronic health records (EHRs) (i.e., e-referrals), lapses in patient follow-up might occur. Although compelling reasons exist why referral coordination should be improved, little is known about which elements of the complex referral coordination process should be targeted for improvement. Using Okhuysen & Bechky's coordination framework, this paper aims to understand the barriers, facilitators, and suggestions for improving communication and coordination of EHR-based referrals in an integrated healthcare system.
Methods
We conducted a qualitative study to understand coordination breakdowns related to e-referrals in an integrated healthcare system and examined work-system factors that affect the timely receipt of subspecialty care. We conducted interviews with seven subject matter experts and six focus groups with a total of 30 PCPs and subspecialists at two tertiary care Department of Veterans Affairs (VA) medical centers. Using techniques from grounded theory and content analysis, we identified organizational themes that affected the referral process.
Results
Four themes emerged: lack of an institutional referral policy, lack of standardization in certain referral procedures, ambiguity in roles and responsibilities, and inadequate resources to adapt and respond to referral requests effectively. Marked differences in PCPs' and subspecialists' communication styles and individual mental models of the referral processes likely precluded the development of a shared mental model to facilitate coordination and successful referral completion. Notably, very few barriers related to the EHR were reported.
Conclusions
Despite facilitating information transfer between PCPs and subspecialists, e-referrals remain prone to coordination breakdowns. Clear referral policies, well-defined roles and responsibilities for key personnel, standardized procedures and communication protocols, and adequate human resources must be in place before implementing an EHR to facilitate referrals.
doi:10.1186/1748-5908-6-84
PMCID: PMC3199858  PMID: 21794109
25.  Understanding the management of electronic test result notifications in the outpatient setting 
Background
Notifying clinicians about abnormal test results through electronic health record (EHR) -based "alert" notifications may not always lead to timely follow-up of patients. We sought to understand barriers, facilitators, and potential interventions for safe and effective management of abnormal test result delivery via electronic alerts.
Methods
We conducted a qualitative study consisting of six 6-8 member focus groups (N = 44) at two large, geographically dispersed Veterans Affairs facilities. Participants included full-time primary care providers, and personnel representing diagnostic services (radiology, laboratory) and information technology. We asked participants to discuss barriers, facilitators, and suggestions for improving timely management and follow-up of abnormal test result notifications and encouraged them to consider technological issues, as well as broader, human-factor-related aspects of EHR use such as organizational, personnel, and workflow.
Results
Providers reported receiving a large number of alerts containing information unrelated to abnormal test results, many of which were believed to be unnecessary. Some providers also reported lacking proficiency in use of certain EHR features that would enable them to manage alerts more efficiently. Suggestions for improvement included improving display and tracking processes for critical alerts in the EHR, redesigning clinical workflow, and streamlining policies and procedures related to test result notification.
Conclusion
Providers perceive several challenges for fail-safe electronic communication and tracking of abnormal test results. A multi-dimensional approach that addresses technology as well as the many non-technological factors we elicited is essential to design interventions to reduce missed test results in EHRs.
doi:10.1186/1472-6947-11-22
PMCID: PMC3100236  PMID: 21486478
Decision Support Systems; Clinical; Automated notification; diagnostic errors; abnormal diagnostic test results; Medical Records Systems; Computerized; patient follow-up; patient safety; health information technology; communication; primary care

Results 1-25 (102)