Patient-generated health data (PGHD) are health-related data created or recorded by patients to inform their self-care and understanding about their own health. PGHD is different from other patient-reported outcome data because the collection of data is patient-driven, not practice- or research-driven. Technical applications for assisting patients to collect PGHD supports self-management activities such as healthy eating and exercise and can be important for preventing and managing disease. Technological innovations (eg, activity trackers) are making it more common for people to collect PGHD, but little is known about how PGHD might be used in outpatient clinics.
The objective of our study was to examine the experiences of health care professionals who use PGHD in outpatient clinics.
We conducted an evaluation of Project HealthDesign Round 2 to synthesize findings from 5 studies funded to test tools designed to help patients collect PGHD and share these data with members of their health care team. We conducted semistructured interviews with 13 Project HealthDesign study team members and 12 health care professionals that participated in these studies. We used an immersion-crystallization approach to analyze data. Our findings provide important information related to health care professionals’ attitudes toward and experiences with using PGHD in a clinical setting.
Health care professionals identified 3 main benefits of PGHD accessibility in clinical settings: (1) deeper insight into a patient’s condition; (2) more accurate patient information, particularly when of clinical relevance; and (3) insight into a patient’s health between clinic visits, enabling revision of care plans for improved health goal achievement, while avoiding unnecessary clinic visits. Study participants also identified 3 areas of consideration when implementing collection and use of PGHD data in clinics: (1) developing practice workflows and protocols related to PGHD collection and use; (2) data storage, accessibility at the point of care, and privacy concerns; and (3) ease of using PGHD data.
PGHD provides value to both patients and health care professionals. However, more research is needed to understand the benefit of using PGHD in clinical care and to identify the strategies and clinic workflow needs for optimizing these tools.
mobile applications; chronic disease, self-management; doctor-patient relations
Accurate display and interpretation of clinical laboratory test results is essential for safe and effective diagnosis and treatment. In an attempt to ascertain how well current electronic health records (EHRs) facilitated these processes, we evaluated the graphical displays of laboratory test results in eight EHRs using objective criteria for optimal graphs based on literature and expert opinion. None of the EHRs met all 11 criteria; the magnitude of deficiency ranged from one EHR meeting 10 of 11 criteria to three EHRs meeting only 5 of 11 criteria. One criterion (i.e., the EHR has a graph with y-axis labels that display both the name of the measured variable and the units of measure) was absent from all EHRs. One EHR system graphed results in reverse chronological order. One EHR system plotted data collected at unequally-spaced points in time using equally-spaced data points, which had the effect of erroneously depicting the visual slope perception between data points. This deficiency could have a significant, negative impact on patient safety. Only two EHR systems allowed users to see, hover-over, or click on a data point to see the precise values of the x–y coordinates. Our study suggests that many current EHR-generated graphs do not meet evidence-based criteria aimed at improving laboratory data comprehension.
diagnostic tests; user computer interface; electronic health records; national health policy
Recently there have been several high-profile ransomware attacks involving hospitals around the world. Ransomware is intended to damage or disable a user’s computer unless the user makes a payment. Once the attack has been launched, users have three options: 1) try to restore their data from backup; 2) pay the ransom; or 3) lose their data. In this manuscript, we discuss a socio-technical approach to address ransomware and outline four overarching steps that organizations can undertake to secure an electronic health record (EHR) system and the underlying computing infrastructure. First, health IT professionals need to ensure adequate system protection by correctly installing and configuring computers and networks that connect them. Next, the health care organizations need to ensure more reliable system defense by implementing user-focused strategies, including simulation and training on correct and complete use of computers and network applications. Concomitantly, the organization needs to monitor computer and application use continuously in an effort to detect suspicious activities and identify and address security problems before they cause harm. Finally, organizations need to respond adequately to and recover quickly from ransomware attacks and take actions to prevent them in future. We also elaborate on recommendations from other authoritative sources, including the National Institute of Standards and Technology (NIST). Similar to approaches to address other complex socio-technical health IT challenges, the responsibility of preventing, mitigating, and recovering from these attacks is shared between health IT professionals and end-users.
Health information technology; electronic health record; socio-technical; cybersecurity; ransomware
To assess problem list completeness using an objective measure across a range of sites, and to identify success factors for problem list completeness.
We conducted a retrospective analysis of electronic health record data and interviews at ten healthcare organizations within the United States, United Kingdom, and Argentina who use a variety of electronic health record systems: four self-developed and six commercial. At each site, we assessed the proportion of patients who have diabetes recorded on their problem list out of all patients with a hemoglobin A1c elevation >= 7.0%, which is diagnostic of diabetes. We then conducted interviews with informatics leaders at the four highest performing sites to determine factors associated with success. Finally, we surveyed all the sites about common practices implemented at the top performing sites to determine whether there was an association between problem list management practices and problem list completeness.
Problem list completeness across the ten sites ranged from 60.2% to 99.4%, with a mean of 78.2%. Financial incentives, problem-oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture were identified as success factors at the four hospitals with problem list completeness at or near 90.0%.
Incomplete problem lists represent a global data integrity problem that could compromise quality of care and put patients at risk. There was a wide range of problem list completeness across the healthcare facilities. Nevertheless, some facilities have achieved high levels of problem list completeness, and it is important to better understand the factors that contribute to success to improve patient safety.
Problem list completeness varies substantially across healthcare facilities. In our review of EHR systems at ten healthcare facilities, we identified six success factors which may be useful for healthcare organizations seeking to improve the quality of their problem list documentation: financial incentives, problem oriented charting, gap reporting, shared responsibility, links to billing codes, and organizational culture.
electronic health records; problem lists; diabetes; quality
Objective Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems.
Methods We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certification Body (ONC-ACB)-certified EHRs. Within each EHR, the authors attempted to implement 3 user-defined rules that utilized the various data and logic elements expected of typical EHRs and that represented clinically important evidenced-based care. The rules were: 1) if a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH; 2) if a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list; and 3) if a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin.
Results Most evaluated EHRs lacked some CDS capabilities; 5 EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to implement any of the rules. One of these did not allow users to customize CDS rules at all. The most frequently found shortcomings included the inability to use laboratory test results in rules, limit rules by time, use advanced Boolean logic, perform actions from the alert interface, and adequately test rules.
Conclusion Significant improvements in the EHR certification and implementation procedures are necessary.
decision support systems; clinical; electronic health records
electronic health records; health information technology; alerts; notifications; medical informatics; primary care; efficiency; patient safety; health services research
To explore the need for, and use of, high-quality, collaborative, clinical knowledge management (CKM) tools and techniques to manage clinical decision support content.
In order to better understand the current state of the art in CKM, we developed a survey of potential CKM tools and techniques. We conducted an exploratory study by querying a convenience sample of respondents about their use of specific practices in CKM.
The following tools and techniques should be priorities in organizations interested in developing successful computer-based provider order entry (CPOE) and clinical decision support (CDS) implementations: 1) A multidisciplinary team responsible for creating and maintaining the clinical content; 2) An external organizational repository of clinical content with web-based viewer that allows anyone in the organization to review it; 3) An online, collaborative, interactive, internet-based tool to facilitate content development; 4) An enterprise-wide tool to maintain the controlled clinical terminology concepts. Even organizations that have been successfully using Computer-based Provider Order Entry with advanced Clinical Decision Support features for well over 15 years are not using all of the CKM tools or practices that we identified.
If we are to further stimulate progress in the area of clinical decision support, we must continue to develop and refine our understanding and use of advanced CKM capabilities.
Health information technology (health IT) has potential to improve patient safety but its implementation and use has led to unintended consequences and new safety concerns. A key challenge to improving safety in health IT-enabled healthcare systems is to develop valid, feasible strategies to measure safety concerns at the intersection of health IT and patient safety. In response to the fundamental conceptual and methodological gaps related to both defining and measuring health IT-related patient safety, we propose a new framework, the Health IT Safety (HITS) measurement framework, to provide a conceptual foundation for health IT-related patient safety measurement, monitoring, and improvement. The HITS framework follows both Continuous Quality Improvement (CQI) and sociotechnical approaches and calls for new measures and measurement activities to address safety concerns in three related domains: 1) concerns that are unique and specific to technology (e.g., to address unsafe health IT related to unavailable or malfunctioning hardware or software); 2) concerns created by the failure to use health IT appropriately or by misuse of health IT (e.g. to reduce nuisance alerts in the electronic health record (EHR)), and 3) the use of health IT to monitor risks, health care processes and outcomes and identify potential safety concerns before they can harm patients (e.g. use EHR-based algorithms to identify patients at risk for medication errors or care delays). The framework proposes to integrate both retrospective and prospective measurement of HIT safety with an organization's existing clinical risk management and safety programs. It aims to facilitate organizational learning, comprehensive 360 degree assessment of HIT safety that includes vendor involvement, refinement of measurement tools and strategies, and shared responsibility to identify problems and implement solutions. A long term framework goal is to enable rigorous measurement that helps achieve the safety benefits of health IT in real-world clinical settings.
Information technology; Patient safety; Performance measures; Quality measurement; Quality improvement methodologies
Computerized clinical decision support (CDS) can help hospitals to improve healthcare. However, CDS can be problematic. The purpose of this study was to discover how the views of clinical stakeholders, CDS content vendors, and EHR vendors are alike or different with respect to challenges in the development, management, and use of CDS.
We conducted ethnographic fieldwork using a Rapid Assessment Process within ten clinical and five health information technology (HIT) vendor organizations. Using an inductive analytical approach, we generated themes from the clinical, content vendor, and electronic health record vendor perspectives and compared them.
The groups share views on the importance of appropriate manpower, careful knowledge management, CDS that fits user workflow, the need for communication among the groups, and for mutual strategizing about the future of CDS. However, views of usability, training, metrics, interoperability, product use, and legal issues differed. Recommendations for improvement include increased collaboration to address legal, manpower, and CDS sharing issues.
The three groups share thinking about many aspects of CDS, but views differ in a number of important respects as well. Until these three groups can reach a mutual understanding of the views of the other stakeholders, and work together, CDS will not reach its potential.
Clinical decision support; Knowledge management; Governance; Rapid assessment process
Comparative Effectiveness Research (CER) has the potential to transform the current healthcare delivery system by identifying the most effective medical and surgical treatments, diagnostic tests, disease prevention methods and ways to deliver care for specific clinical conditions. To be successful, such research requires the identification, capture, aggregation, integration, and analysis of disparate data sources held by different institutions with diverse representations of the relevant clinical events. In an effort to address these diverse demands, there have been multiple new designs and implementations of informatics platforms that provide access to electronic clinical data and the governance infrastructure required for inter-institutional CER. The goal of this manuscript is to help investigators understand why these informatics platforms are required and to compare and contrast six, large-scale, recently funded, CER-focused informatics platform development efforts. We utilized an 8-dimension, socio-technical model of health information technology use to help guide our work. We identified six generic steps that are necessary in any distributed, multi-institutional CER project: data identification, extraction, modeling, aggregation, analysis, and dissemination. We expect that over the next several years these projects will provide answers to many important, and heretofore unanswerable, clinical research questions.
Methods; Comparative Effectiveness Research; Organization and Administration; Medical Informatics; Methods
Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements.
We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review.
We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs.
Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases.
A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.
Electronic health records; Crowdsourcing; Knowledge bases; Medical records; Problem-oriented
Despite its promise, recent literature has revealed possible safety hazards of health information technology (HIT) use. The Office of the National Coordinator for HIT recently sponsored an Institute of Medicine committee to synthesize evidence and experience from the field on how HIT affects patient safety. To lay the groundwork for defining, measuring, and analyzing HIT-related safety hazards, we propose that Health information technology-related error occurs anytime HIT is unavailable for use, malfunctions during use, is used incorrectly by someone, or when HIT interacts with another system component incorrectly, resulting in data being lost or incorrectly entered, displayed, or transmitted. These errors, or the decisions that result from them, significantly increase the risk of adverse events and patient harm. In this paper, we describe how a socio-technical approach can be used to understand the complex origins of HIT errors, which may have roots in rapidly evolving technological, professional, organizational, and policy initiatives.
Electronic Health Records; Health Information Technology; Patient Safety; Errors
We conducted a meta-synthesis of five different studies that developed, tested, and implemented new technologies for the purpose of collecting Observations of Daily Living (ODL). From this synthesis, we developed a model to explain user motivation as it relates to ODL collection. We describe this model that includes six factors that motivate patients’ collection of ODL data: usability, illness experience, relevance of ODLs, information technology infrastructure, degree of burden, and emotional activation. We show how these factors can act as barriers or facilitators to the collection of ODL data and how interacting with care professionals and sharing ODL data may also influence ODL collection, health-related awareness, and behavior change. The model we developed and used to explain ODL collection can be helpful to researchers and designers who study and develop new, personal health technologies to empower people to improve their health.
Observations of daily living (ODLs); mobile health tracking; behavior change; patient/provider communication; smart phones; user burden; user motivation
Electronic health records (EHRs) facilitate several innovations capable of reforming health care. Despite their promise, many currently unanswered legal, ethical, and financial questions threaten the widespread adoption and use of EHRs. Key legal dilemmas that must be addressed in the near-term pertain to the extent of clinicians' responsibilities for reviewing the entire computer-accessible clinical synopsis from multiple clinicians and institutions, the liabilities posed by overriding clinical decision support warnings and alerts, and mechanisms for clinicians to publically report potential EHR safety issues. Ethical dilemmas that need additional discussion relate to opt-out provisions that exclude patients from electronic record storage, sale of deidentified patient data by EHR vendors, adolescent control of access to their data, and use of electronic data repositories to redesign the nation's health care delivery and payment mechanisms on the basis of statistical analyses. Finally, one overwhelming financial question is who should pay for EHR implementation because most users and current owners of these systems will not receive the majority of benefits. The authors recommend that key stakeholders begin discussing these issues in a national forum. These actions can help identify and prioritize solutions to the key legal, ethical, and financial dilemmas discussed, so that widespread, safe, effective, interoperable EHRs can help transform health care.
electronic health records; ethics; medical; confidentiality
On September 30th, 2014, the Centers for Disease Control and Prevention (CDC) confirmed the first travel-associated case of US Ebola in Dallas, TX. This case exposed two of the greatest concerns in patient safety in the US outpatient health care system: misdiagnosis and ineffective use of electronic health records (EHRs). The case received widespread media attention highlighting failures in disaster management, infectious disease control, national security, and emergency department (ED) care. In addition, an error in making a correct and timely Ebola diagnosis on initial ED presentation brought diagnostic decision-making vulnerabilities in the EHR era into the public eye. In this paper, we use this defining “teachable moment” to highlight the public health challenge of diagnostic errors and discuss the effective use of EHRs in the diagnostic process. We analyze the case to discuss several missed opportunities and outline key challenges and opportunities facing diagnostic decision-making in EHR-enabled health care. It is important to recognize the reality that EHRs suffer from major usability and inter-operability issues, but also to acknowledge that they are only tools and not a replacement for basic history-taking, examination skills, and critical thinking. While physicians and health care organizations ultimately need to own the responsibility for addressing diagnostic errors, several national-level initiatives can help, including working with software developers to improve EHR usability. Multifaceted approaches that account for both technical and non-technical factors will be needed. Ebola US Patient Zero reminds us that in certain cases, a single misdiagnosis can have widespread and costly implications for public health.
cognition; decision-making; diagnostic error; Ebola; electronic medical records; health information technology; human factors; misdiagnosis; patient safety
Conceptual models have been developed to address challenges inherent in studying health information technology (HIT). This manuscript introduces an 8-dimensional model specifically designed to address the socio-technical challenges involved in design, development, implementation, use, and evaluation of HIT within complex adaptive healthcare systems. The 8 dimensions are not independent, sequential, or hierarchical, but rather are interdependent and interrelated concepts similar to compositions of other complex adaptive systems. Hardware and software computing infrastructure refers to equipment and software used to power, support, and operate clinical applications and devices. Clinical content refers to textual or numeric data and images that constitute the “language” of clinical applications. The human computer interface includes all aspects of the computer that users can see, touch, or hear as they interact with it. People refers to everyone who interacts in some way with the system, from developer to end-user, including potential patient-users. Workflow and communication are the processes or steps involved in assuring that patient care tasks are carried out effectively. Two additional dimensions of the model are internal organizational features (e.g., policies, procedures, and culture) and external rules and regulations, both of which may facilitate or constrain many aspects of the preceding dimensions. The final dimension is measurement and monitoring, which refers to the process of measuring and evaluating both intended and unintended consequences of HIT implementation and use. We illustrate how our model has been successfully applied in real-world complex adaptive settings to understand and improve HIT applications at various stages of development and implementation.
The US FDA has been collecting information on medical devices involved in significant adverse advents since 1984. These reports have been used by researchers to advise clinicians on potential risks and complications of using these devices.
Research adverse events related to the use of Clinical Information Systems (CIS) as reported in FDA databases.
Three large, national, adverse event medical device databases were examined for reports pertaining to CIS.
One hundred and twenty unique reports (from over 1.4 million reports) were found, representing 32 manufacturers. The manifestations of these adverse events included: missing or incorrect data, data displayed for the wrong patient, chaos during system downtime and system unavailable for use. Analysis of these reports illustrated events associated with system design, implementation, use, and support.
The identified causes can be used by manufacturers to improve their products and by clinical facilities and providers to adjust their workflow and implementation processes appropriately. The small number of reports found indicates a need to raise awareness regarding publicly available tools for documenting problems with CIS and for additional reporting and dialog between manufacturers, organizations, and users.
Electronic Health Records; Information Systems; Mandatory Reporting; Medical Errors; United States Food and Drug Administration
Intensive care units (ICUs) become more complicated each day; the number of devices to monitor various aspects of a patient's status continues to increase. Intelligent monitors attempt to reduce this complexity by interpreting the data and presenting a high-level summary of a patient's condition. We propose a parallel software architecture for constructing intelligent medical monitors, the process trellis; we contrast the process trellis to other software architectures that have been used for heuristic medical programs: blackboards and production rules. The process trellis is an explicitly parallel structure, and therefore can take advantage of the performance gains available from parallel computer hardware. Its use does not, however, presuppose any expertise in parallel programming. We are currently building an Intelligent Cardiovascular Monitor (ICM) using the process trellis. We describe the ICM and the use of the process trellis architecture in its construction.
Larry Weed, MD is widely known as the father of the problem-oriented medical record and inventor of the now-ubiquitous SOAP (subjective/objective/assessment/plan) note, for developing an electronic health record system (Problem-Oriented Medical Information System, PROMIS), and for founding a company (since acquired), which developed problem-knowledge couplers. However, Dr Weed's vision for medicine goes far beyond software—over the course of his storied career, he has relentlessly sought to bring the scientific method to medical practice and, where necessary, to point out shortcomings in the system and advocate for change. In this oral history, Dr Weed describes, in his own words, the arcs of his long career and the work that remains to be done.
Oral History; Problem List; Problem-Oriented Medical Record
To determine what “average” clinicians in organizations that were about to implement Computer-based Provider Order Entry (CPOE) were expecting to occur, we conducted an open-ended, semi-structured survey at three community hospitals.
We created an open-ended, semi-structured, interview survey template that we customized for each organization. This interview-based survey was designed to be administered orally to clinicians and take approximately five minutes to complete, although clinicians were allowed to discuss as many advantages or disadvantages of the impending system roll-out as they wanted to.
Our survey findings did not reveal any overly negative, critical, problematic, or striking sets of concerns. However, from the standpoint of unintended consequences, we found that clinicians were anticipating only a few of the events, emotions, and process changes that are likely to result from CPOE.
The results of such an open-ended survey may prove useful in helping CPOE leaders to understand user perceptions and predictions about CPOE, because it can expose issues about which more communication, or discussion, is needed. Using the survey, implementation strategies and management techniques outlined in this paper, any chief information officer (CIO) or chief medical information officer (CMIO) should be able to adequately assess their organization's CPOE readiness, make the necessary mid-course corrections, and be prepared to deal with the currently identified unintended consequences of CPOE should they occur.
Medical Order Entry Systems; Ethnology; Hospitals, Community; Medical Informatics
There is a pressing need for high-quality, effective means of designing, developing, presenting, implementing, evaluating, and maintaining all types of clinical decision support capabilities for clinicians, patients and consumers. Using an iterative, consensus-building process we identified a rank-ordered list of the top 10 grand challenges in clinical decision support. This list was created to educate and inspire researchers, developers, funders, and policy-makers. The list of challenges in order of importance that they be solved if patients and organizations are to begin realizing the fullest benefits possible of these systems consists of: Improve the human-computer interface; Disseminate best practices in CDS design, development, and implementation; Summarize patient-level information; Prioritize and filter recommendations to the user; Create an architecture for sharing executable CDS modules and services; Combine recommendations for patients with co-morbidities; Prioritize CDS content development and implementation; Create internet-accessible clinical decision support repositories; Use freetext information to drive clinical decision support; Mine large clinical databases to create new CDS. Identification of solutions to these challenges is critical if clinical decision support is to achieve its potential and improve the quality, safety and efficiency of healthcare.
Background and purpose
Target volumes and organs-at-risk (OARs) for radiotherapy (RT) planning are manually defined, which is a tedious and inaccurate process. We sought to assess the feasibility, time reduction, and acceptability of an atlas-based autosegmentation (AS) compared to manual segmentation (MS) of OARs.
Materials and methods
A commercial platform generated 16 OARs. Resident physicians were randomly assigned to modify AS OAR (AS + R) or to draw MS OAR followed by attending physician correction. Dice similarity coefficient (DSC) was used to measure overlap between groups compared with attending approved OARs (DSC = 1 means perfect overlap). 40 cases were segmented.
Mean ± SD segmentation time in the AS + R group was 19.7 ± 8.0 min, compared to 28.5 ± 8.0 min in the MS cohort, amounting to a 30.9% time reduction (Wilcoxon p < 0.01). For each OAR, AS DSC was statistically different from both AS + R and MS ROIs (all Steel–Dwass p < 0.01) except the spinal cord and the mandible, suggesting oversight of AS/MS processes is required; AS + R and MS DSCs were non-different. AS compared to attending approved OAR DSCs varied considerably, with a chiasm mean ± SD DSC of 0.37 ± 0.32 and brainstem of 0.97 ± 0.03.
Autosegmentation provides a time savings in head and neck regions of interest generation. However, attending physician approval remains vital.
Atlas-based autosegmentation; Normal tissue; Autocontouring; Head and neck; Automatic segmentation; Organs-at-risk