PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1440621)

Clipboard (0)
None

Related Articles

1.  Internet-Based Device-Assisted Remote Monitoring of Cardiovascular Implantable Electronic Devices 
Executive Summary
Objective
The objective of this Medical Advisory Secretariat (MAS) report was to conduct a systematic review of the available published evidence on the safety, effectiveness, and cost-effectiveness of Internet-based device-assisted remote monitoring systems (RMSs) for therapeutic cardiac implantable electronic devices (CIEDs) such as pacemakers (PMs), implantable cardioverter-defibrillators (ICDs), and cardiac resynchronization therapy (CRT) devices. The MAS evidence-based review was performed to support public financing decisions.
Clinical Need: Condition and Target Population
Sudden cardiac death (SCD) is a major cause of fatalities in developed countries. In the United States almost half a million people die of SCD annually, resulting in more deaths than stroke, lung cancer, breast cancer, and AIDS combined. In Canada each year more than 40,000 people die from a cardiovascular related cause; approximately half of these deaths are attributable to SCD.
Most cases of SCD occur in the general population typically in those without a known history of heart disease. Most SCDs are caused by cardiac arrhythmia, an abnormal heart rhythm caused by malfunctions of the heart’s electrical system. Up to half of patients with significant heart failure (HF) also have advanced conduction abnormalities.
Cardiac arrhythmias are managed by a variety of drugs, ablative procedures, and therapeutic CIEDs. The range of CIEDs includes pacemakers (PMs), implantable cardioverter-defibrillators (ICDs), and cardiac resynchronization therapy (CRT) devices. Bradycardia is the main indication for PMs and individuals at high risk for SCD are often treated by ICDs.
Heart failure (HF) is also a significant health problem and is the most frequent cause of hospitalization in those over 65 years of age. Patients with moderate to severe HF may also have cardiac arrhythmias, although the cause may be related more to heart pump or haemodynamic failure. The presence of HF, however, increases the risk of SCD five-fold, regardless of aetiology. Patients with HF who remain highly symptomatic despite optimal drug therapy are sometimes also treated with CRT devices.
With an increasing prevalence of age-related conditions such as chronic HF and the expanding indications for ICD therapy, the rate of ICD placement has been dramatically increasing. The appropriate indications for ICD placement, as well as the rate of ICD placement, are increasingly an issue. In the United States, after the introduction of expanded coverage of ICDs, a national ICD registry was created in 2005 to track these devices. A recent survey based on this national ICD registry reported that 22.5% (25,145) of patients had received a non-evidence based ICD and that these patients experienced significantly higher in-hospital mortality and post-procedural complications.
In addition to the increased ICD device placement and the upfront device costs, there is the need for lifelong follow-up or surveillance, placing a significant burden on patients and device clinics. In 2007, over 1.6 million CIEDs were implanted in Europe and the United States, which translates to over 5.5 million patient encounters per year if the recommended follow-up practices are considered. A safe and effective RMS could potentially improve the efficiency of long-term follow-up of patients and their CIEDs.
Technology
In addition to being therapeutic devices, CIEDs have extensive diagnostic abilities. All CIEDs can be interrogated and reprogrammed during an in-clinic visit using an inductive programming wand. Remote monitoring would allow patients to transmit information recorded in their devices from the comfort of their own homes. Currently most ICD devices also have the potential to be remotely monitored. Remote monitoring (RM) can be used to check system integrity, to alert on arrhythmic episodes, and to potentially replace in-clinic follow-ups and manage disease remotely. They do not currently have the capability of being reprogrammed remotely, although this feature is being tested in pilot settings.
Every RMS is specifically designed by a manufacturer for their cardiac implant devices. For Internet-based device-assisted RMSs, this customization includes details such as web application, multiplatform sensors, custom algorithms, programming information, and types and methods of alerting patients and/or physicians. The addition of peripherals for monitoring weight and pressure or communicating with patients through the onsite communicators also varies by manufacturer. Internet-based device-assisted RMSs for CIEDs are intended to function as a surveillance system rather than an emergency system.
Health care providers therefore need to learn each application, and as more than one application may be used at one site, multiple applications may need to be reviewed for alarms. All RMSs deliver system integrity alerting; however, some systems seem to be better geared to fast arrhythmic alerting, whereas other systems appear to be more intended for remote follow-up or supplemental remote disease management. The different RMSs may therefore have different impacts on workflow organization because of their varying frequency of interrogation and methods of alerts. The integration of these proprietary RM web-based registry systems with hospital-based electronic health record systems has so far not been commonly implemented.
Currently there are 2 general types of RMSs: those that transmit device diagnostic information automatically and without patient assistance to secure Internet-based registry systems, and those that require patient assistance to transmit information. Both systems employ the use of preprogrammed alerts that are either transmitted automatically or at regular scheduled intervals to patients and/or physicians.
The current web applications, programming, and registry systems differ greatly between the manufacturers of transmitting cardiac devices. In Canada there are currently 4 manufacturers—Medtronic Inc., Biotronik, Boston Scientific Corp., and St Jude Medical Inc.—which have regulatory approval for remote transmitting CIEDs. Remote monitoring systems are proprietary to the manufacturer of the implant device. An RMS for one device will not work with another device, and the RMS may not work with all versions of the manufacturer’s devices.
All Internet-based device-assisted RMSs have common components. The implanted device is equipped with a micro-antenna that communicates with a small external device (at bedside or wearable) commonly known as the transmitter. Transmitters are able to interrogate programmed parameters and diagnostic data stored in the patients’ implant device. The information transfer to the communicator can occur at preset time intervals with the participation of the patient (waving a wand over the device) or it can be sent automatically (wirelessly) without their participation. The encrypted data are then uploaded to an Internet-based database on a secure central server. The data processing facilities at the central database, depending on the clinical urgency, can trigger an alert for the physician(s) that can be sent via email, fax, text message, or phone. The details are also posted on the secure website for viewing by the physician (or their delegate) at their convenience.
Research Questions
The research directions and specific research questions for this evidence review were as follows:
To identify the Internet-based device-assisted RMSs available for follow-up of patients with therapeutic CIEDs such as PMs, ICDs, and CRT devices.
To identify the potential risks, operational issues, or organizational issues related to Internet-based device-assisted RM for CIEDs.
To evaluate the safety, acceptability, and effectiveness of Internet-based device-assisted RMSs for CIEDs such as PMs, ICDs, and CRT devices.
To evaluate the safety, effectiveness, and cost-effectiveness of Internet-based device-assisted RMSs for CIEDs compared to usual outpatient in-office monitoring strategies.
To evaluate the resource implications or budget impact of RMSs for CIEDs in Ontario, Canada.
Research Methods
Literature Search
The review included a systematic review of published scientific literature and consultations with experts and manufacturers of all 4 approved RMSs for CIEDs in Canada. Information on CIED cardiac implant clinics was also obtained from Provincial Programs, a division within the Ministry of Health and Long-Term Care with a mandate for cardiac implant specialty care. Various administrative databases and registries were used to outline the current clinical follow-up burden of CIEDs in Ontario. The provincial population-based ICD database developed and maintained by the Institute for Clinical Evaluative Sciences (ICES) was used to review the current follow-up practices with Ontario patients implanted with ICD devices.
Search Strategy
A literature search was performed on September 21, 2010 using OVID MEDLINE, MEDLINE In-Process and Other Non-Indexed Citations, EMBASE, the Cumulative Index to Nursing & Allied Health Literature (CINAHL), the Cochrane Library, and the International Agency for Health Technology Assessment (INAHTA) for studies published from 1950 to September 2010. Search alerts were generated and reviewed for additional relevant literature until December 31, 2010. Abstracts were reviewed by a single reviewer and, for those studies meeting the eligibility criteria full-text articles were obtained. Reference lists were also examined for any additional relevant studies not identified through the search.
Inclusion Criteria
published between 1950 and September 2010;
English language full-reports and human studies;
original reports including clinical evaluations of Internet-based device-assisted RMSs for CIEDs in clinical settings;
reports including standardized measurements on outcome events such as technical success, safety, effectiveness, cost, measures of health care utilization, morbidity, mortality, quality of life or patient satisfaction;
randomized controlled trials (RCTs), systematic reviews and meta-analyses, cohort and controlled clinical studies.
Exclusion Criteria
non-systematic reviews, letters, comments and editorials;
reports not involving standardized outcome events;
clinical reports not involving Internet-based device assisted RM systems for CIEDs in clinical settings;
reports involving studies testing or validating algorithms without RM;
studies with small samples (<10 subjects).
Outcomes of Interest
The outcomes of interest included: technical outcomes, emergency department visits, complications, major adverse events, symptoms, hospital admissions, clinic visits (scheduled and/or unscheduled), survival, morbidity (disease progression, stroke, etc.), patient satisfaction, and quality of life.
Summary of Findings
The MAS evidence review was performed to review available evidence on Internet-based device-assisted RMSs for CIEDs published until September 2010. The search identified 6 systematic reviews, 7 randomized controlled trials, and 19 reports for 16 cohort studies—3 of these being registry-based and 4 being multi-centered. The evidence is summarized in the 3 sections that follow.
1. Effectiveness of Remote Monitoring Systems of CIEDs for Cardiac Arrhythmia and Device Functioning
In total, 15 reports on 13 cohort studies involving investigations with 4 different RMSs for CIEDs in cardiology implant clinic groups were identified in the review. The 4 RMSs were: Care Link Network® (Medtronic Inc,, Minneapolis, MN, USA); Home Monitoring® (Biotronic, Berlin, Germany); House Call 11® (St Jude Medical Inc., St Pauls, MN, USA); and a manufacturer-independent RMS. Eight of these reports were with the Home Monitoring® RMS (12,949 patients), 3 were with the Care Link® RMS (167 patients), 1 was with the House Call 11® RMS (124 patients), and 1 was with a manufacturer-independent RMS (44 patients). All of the studies, except for 2 in the United States, (1 with Home Monitoring® and 1 with House Call 11®), were performed in European countries.
The RMSs in the studies were evaluated with different cardiac implant device populations: ICDs only (6 studies), ICD and CRT devices (3 studies), PM and ICD and CRT devices (4 studies), and PMs only (2 studies). The patient populations were predominately male (range, 52%–87%) in all studies, with mean ages ranging from 58 to 76 years. One study population was unique in that RMSs were evaluated for ICDs implanted solely for primary prevention in young patients (mean age, 44 years) with Brugada syndrome, which carries an inherited increased genetic risk for sudden heart attack in young adults.
Most of the cohort studies reported on the feasibility of RMSs in clinical settings with limited follow-up. In the short follow-up periods of the studies, the majority of the events were related to detection of medical events rather than system configuration or device abnormalities. The results of the studies are summarized below:
The interrogation of devices on the web platform, both for continuous and scheduled transmissions, was significantly quicker with remote follow-up, both for nurses and physicians.
In a case-control study focusing on a Brugada population–based registry with patients followed-up remotely, there were significantly fewer outpatient visits and greater detection of inappropriate shocks. One death occurred in the control group not followed remotely and post-mortem analysis indicated early signs of lead failure prior to the event.
Two studies examined the role of RMSs in following ICD leads under regulatory advisory in a European clinical setting and noted:
– Fewer inappropriate shocks were administered in the RM group.
– Urgent in-office interrogations and surgical revisions were performed within 12 days of remote alerts.
– No signs of lead fracture were detected at in-office follow-up; all were detected at remote follow-up.
Only 1 study reported evaluating quality of life in patients followed up remotely at 3 and 6 months; no values were reported.
Patient satisfaction was evaluated in 5 cohort studies, all in short term follow-up: 1 for the Home Monitoring® RMS, 3 for the Care Link® RMS, and 1 for the House Call 11® RMS.
– Patients reported receiving a sense of security from the transmitter, a good relationship with nurses and physicians, positive implications for their health, and satisfaction with RM and organization of services.
– Although patients reported that the system was easy to implement and required less than 10 minutes to transmit information, a variable proportion of patients (range, 9% 39%) reported that they needed the assistance of a caregiver for their transmission.
– The majority of patients would recommend RM to other ICD patients.
– Patients with hearing or other physical or mental conditions hindering the use of the system were excluded from studies, but the frequency of this was not reported.
Physician satisfaction was evaluated in 3 studies, all with the Care Link® RMS:
– Physicians reported an ease of use and high satisfaction with a generally short-term use of the RMS.
– Physicians reported being able to address the problems in unscheduled patient transmissions or physician initiated transmissions remotely, and were able to handle the majority of the troubleshooting calls remotely.
– Both nurses and physicians reported a high level of satisfaction with the web registry system.
2. Effectiveness of Remote Monitoring Systems in Heart Failure Patients for Cardiac Arrhythmia and Heart Failure Episodes
Remote follow-up of HF patients implanted with ICD or CRT devices, generally managed in specialized HF clinics, was evaluated in 3 cohort studies: 1 involved the Home Monitoring® RMS and 2 involved the Care Link® RMS. In these RMSs, in addition to the standard diagnostic features, the cardiac devices continuously assess other variables such as patient activity, mean heart rate, and heart rate variability. Intra-thoracic impedance, a proxy measure for lung fluid overload, was also measured in the Care Link® studies. The overall diagnostic performance of these measures cannot be evaluated, as the information was not reported for patients who did not experience intra-thoracic impedance threshold crossings or did not undergo interventions. The trial results involved descriptive information on transmissions and alerts in patients experiencing high morbidity and hospitalization in the short study periods.
3. Comparative Effectiveness of Remote Monitoring Systems for CIEDs
Seven RCTs were identified evaluating RMSs for CIEDs: 2 were for PMs (1276 patients) and 5 were for ICD/CRT devices (3733 patients). Studies performed in the clinical setting in the United States involved both the Care Link® RMS and the Home Monitoring® RMS, whereas all studies performed in European countries involved only the Home Monitoring® RMS.
3A. Randomized Controlled Trials of Remote Monitoring Systems for Pacemakers
Two trials, both multicenter RCTs, were conducted in different countries with different RMSs and study objectives. The PREFER trial was a large trial (897 patients) performed in the United States examining the ability of Care Link®, an Internet-based remote PM interrogation system, to detect clinically actionable events (CAEs) sooner than the current in-office follow-up supplemented with transtelephonic monitoring transmissions, a limited form of remote device interrogation. The trial results are summarized below:
In the 375-day mean follow-up, 382 patients were identified with at least 1 CAE—111 patients in the control arm and 271 in the remote arm.
The event rate detected per patient for every type of CAE, except for loss of atrial capture, was higher in the remote arm than the control arm.
The median time to first detection of CAEs (4.9 vs. 6.3 months) was significantly shorter in the RMS group compared to the control group (P < 0.0001).
Additionally, only 2% (3/190) of the CAEs in the control arm were detected during a transtelephonic monitoring transmission (the rest were detected at in-office follow-ups), whereas 66% (446/676) of the CAEs were detected during remote interrogation.
The second study, the OEDIPE trial, was a smaller trial (379 patients) performed in France evaluating the ability of the Home Monitoring® RMS to shorten PM post-operative hospitalization while preserving the safety of conventional management of longer hospital stays.
Implementation and operationalization of the RMS was reported to be successful in 91% (346/379) of the patients and represented 8144 transmissions.
In the RM group 6.5% of patients failed to send messages (10 due to improper use of the transmitter, 2 with unmanageable stress). Of the 172 patients transmitting, 108 patients sent a total of 167 warnings during the trial, with a greater proportion of warnings being attributed to medical rather than technical causes.
Forty percent had no warning message transmission and among these, 6 patients experienced a major adverse event and 1 patient experienced a non-major adverse event. Of the 6 patients having a major adverse event, 5 contacted their physician.
The mean medical reaction time was faster in the RM group (6.5 ± 7.6 days vs. 11.4 ± 11.6 days).
The mean duration of hospitalization was significantly shorter (P < 0.001) for the RM group than the control group (3.2 ± 3.2 days vs. 4.8 ± 3.7 days).
Quality of life estimates by the SF-36 questionnaire were similar for the 2 groups at 1-month follow-up.
3B. Randomized Controlled Trials Evaluating Remote Monitoring Systems for ICD or CRT Devices
The 5 studies evaluating the impact of RMSs with ICD/CRT devices were conducted in the United States and in European countries and involved 2 RMSs—Care Link® and Home Monitoring ®. The objectives of the trials varied and 3 of the trials were smaller pilot investigations.
The first of the smaller studies (151 patients) evaluated patient satisfaction, achievement of patient outcomes, and the cost-effectiveness of the Care Link® RMS compared to quarterly in-office device interrogations with 1-year follow-up.
Individual outcomes such as hospitalizations, emergency department visits, and unscheduled clinic visits were not significantly different between the study groups.
Except for a significantly higher detection of atrial fibrillation in the RM group, data on ICD detection and therapy were similar in the study groups.
Health-related quality of life evaluated by the EuroQoL at 6-month or 12-month follow-up was not different between study groups.
Patients were more satisfied with their ICD care in the clinic follow-up group than in the remote follow-up group at 6-month follow-up, but were equally satisfied at 12- month follow-up.
The second small pilot trial (20 patients) examined the impact of RM follow-up with the House Call 11® system on work schedules and cost savings in patients randomized to 2 study arms varying in the degree of remote follow-up.
The total time including device interrogation, transmission time, data analysis, and physician time required was significantly shorter for the RM follow-up group.
The in-clinic waiting time was eliminated for patients in the RM follow-up group.
The physician talk time was significantly reduced in the RM follow-up group (P < 0.05).
The time for the actual device interrogation did not differ in the study groups.
The third small trial (115 patients) examined the impact of RM with the Home Monitoring® system compared to scheduled trimonthly in-clinic visits on the number of unplanned visits, total costs, health-related quality of life (SF-36), and overall mortality.
There was a 63.2% reduction in in-office visits in the RM group.
Hospitalizations or overall mortality (values not stated) were not significantly different between the study groups.
Patient-induced visits were higher in the RM group than the in-clinic follow-up group.
The TRUST Trial
The TRUST trial was a large multicenter RCT conducted at 102 centers in the United States involving the Home Monitoring® RMS for ICD devices for 1450 patients. The primary objectives of the trial were to determine if remote follow-up could be safely substituted for in-office clinic follow-up (3 in-office visits replaced) and still enable earlier physician detection of clinically actionable events.
Adherence to the protocol follow-up schedule was significantly higher in the RM group than the in-office follow-up group (93.5% vs. 88.7%, P < 0.001).
Actionability of trimonthly scheduled checks was low (6.6%) in both study groups. Overall, actionable causes were reprogramming (76.2%), medication changes (24.8%), and lead/system revisions (4%), and these were not different between the 2 study groups.
The overall mean number of in-clinic and hospital visits was significantly lower in the RM group than the in-office follow-up group (2.1 per patient-year vs. 3.8 per patient-year, P < 0.001), representing a 45% visit reduction at 12 months.
The median time from onset of first arrhythmia to physician evaluation was significantly shorter (P < 0.001) in the RM group than in the in-office follow-up group for all arrhythmias (1 day vs. 35.5 days).
The median time to detect clinically asymptomatic arrhythmia events—atrial fibrillation (AF), ventricular fibrillation (VF), ventricular tachycardia (VT), and supra-ventricular tachycardia (SVT)—was also significantly shorter (P < 0.001) in the RM group compared to the in-office follow-up group (1 day vs. 41.5 days) and was significantly quicker for each of the clinical arrhythmia events—AF (5.5 days vs. 40 days), VT (1 day vs. 28 days), VF (1 day vs. 36 days), and SVT (2 days vs. 39 days).
System-related problems occurred infrequently in both groups—in 1.5% of patients (14/908) in the RM group and in 0.7% of patients (3/432) in the in-office follow-up group.
The overall adverse event rate over 12 months was not significantly different between the 2 groups and individual adverse events were also not significantly different between the RM group and the in-office follow-up group: death (3.4% vs. 4.9%), stroke (0.3% vs. 1.2%), and surgical intervention (6.6% vs. 4.9%), respectively.
The 12-month cumulative survival was 96.4% (95% confidence interval [CI], 95.5%–97.6%) in the RM group and 94.2% (95% confidence interval [CI], 91.8%–96.6%) in the in-office follow-up group, and was not significantly different between the 2 groups (P = 0.174).
The CONNECT Trial
The CONNECT trial, another major multicenter RCT, involved the Care Link® RMS for ICD/CRT devices in a15-month follow-up study of 1,997 patients at 133 sites in the United States. The primary objective of the trial was to determine whether automatically transmitted physician alerts decreased the time from the occurrence of clinically relevant events to medical decisions. The trial results are summarized below:
Of the 575 clinical alerts sent in the study, 246 did not trigger an automatic physician alert. Transmission failures were related to technical issues such as the alert not being programmed or not being reset, and/or a variety of patient factors such as not being at home and the monitor not being plugged in or set up.
The overall mean time from the clinically relevant event to the clinical decision was significantly shorter (P < 0.001) by 17.4 days in the remote follow-up group (4.6 days for 172 patients) than the in-office follow-up group (22 days for 145 patients).
– The median time to a clinical decision was shorter in the remote follow-up group than in the in-office follow-up group for an AT/AF burden greater than or equal to 12 hours (3 days vs. 24 days) and a fast VF rate greater than or equal to 120 beats per minute (4 days vs. 23 days).
Although infrequent, similar low numbers of events involving low battery and VF detection/therapy turned off were noted in both groups. More alerts, however, were noted for out-of-range lead impedance in the RM group (18 vs. 6 patients), and the time to detect these critical events was significantly shorter in the RM group (same day vs. 17 days).
Total in-office clinic visits were reduced by 38% from 6.27 visits per patient-year in the in-office follow-up group to 3.29 visits per patient-year in the remote follow-up group.
Health care utilization visits (N = 6,227) that included cardiovascular-related hospitalization, emergency department visits, and unscheduled clinic visits were not significantly higher in the remote follow-up group.
The overall mean length of hospitalization was significantly shorter (P = 0.002) for those in the remote follow-up group (3.3 days vs. 4.0 days) and was shorter both for patients with ICD (3.0 days vs. 3.6 days) and CRT (3.8 days vs. 4.7 days) implants.
The mortality rate between the study arms was not significantly different between the follow-up groups for the ICDs (P = 0.31) or the CRT devices with defribillator (P = 0.46).
Conclusions
There is limited clinical trial information on the effectiveness of RMSs for PMs. However, for RMSs for ICD devices, multiple cohort studies and 2 large multicenter RCTs demonstrated feasibility and significant reductions in in-office clinic follow-ups with RMSs in the first year post implantation. The detection rates of clinically significant events (and asymptomatic events) were higher, and the time to a clinical decision for these events was significantly shorter, in the remote follow-up groups than in the in-office follow-up groups. The earlier detection of clinical events in the remote follow-up groups, however, was not associated with lower morbidity or mortality rates in the 1-year follow-up. The substitution of almost all the first year in-office clinic follow-ups with RM was also not associated with an increased health care utilization such as emergency department visits or hospitalizations.
The follow-up in the trials was generally short-term, up to 1 year, and was a more limited assessment of potential longer term device/lead integrity complications or issues. None of the studies compared the different RMSs, particularly the different RMSs involving patient-scheduled transmissions or automatic transmissions. Patients’ acceptance of and satisfaction with RM were reported to be high, but the impact of RM on patients’ health-related quality of life, particularly the psychological aspects, was not evaluated thoroughly. Patients who are not technologically competent, having hearing or other physical/mental impairments, were identified as potentially disadvantaged with remote surveillance. Cohort studies consistently identified subgroups of patients who preferred in-office follow-up. The evaluation of costs and workflow impact to the health care system were evaluated in European or American clinical settings, and only in a limited way.
Internet-based device-assisted RMSs involve a new approach to monitoring patients, their disease progression, and their CIEDs. Remote monitoring also has the potential to improve the current postmarket surveillance systems of evolving CIEDs and their ongoing hardware and software modifications. At this point, however, there is insufficient information to evaluate the overall impact to the health care system, although the time saving and convenience to patients and physicians associated with a substitution of in-office follow-up by RM is more certain. The broader issues surrounding infrastructure, impacts on existing clinical care systems, and regulatory concerns need to be considered for the implementation of Internet-based RMSs in jurisdictions involving different clinical practices.
PMCID: PMC3377571  PMID: 23074419
2.  Clinical decision making in a high-risk primary care environment: a qualitative study in the UK 
BMJ Open  2012;2(1):e000414.
Objective
Examine clinical reasoning and decision making in an out of hours (OOH) primary care setting to gain insights into how general practitioners (GPs) make clinical decisions and manage risk in this environment.
Design
Semi-structured interviews using open-ended questions.
Setting
A 2-month qualitative interview study conducted in Oxfordshire, UK.
Participants
21 GPs working in OOH primary care.
Results
The most powerful themes to emerge related to dealing with urgent potentially high-risk cases, keeping patients safe and responding to their needs, while trying to keep patients out of hospital and the concept of ‘fire fighting’. There were a number of well-defined characteristics that GPs reported making presentations easy or difficult to deal with. Severely ill patients were straightforward, while the older people, with complex multisystem diseases, were often difficult. GPs stopped collecting clinical information and came to clinical decisions when high-risk disease and severe illness requiring hospital attention has been excluded; they had responded directly to the patient's needs and there was a reliable safety net in place. Learning points that GPs identified as important for trainees in the OOH setting included the importance of developing rapport in spite of time pressures, learning to deal with uncertainty and learning about common presentations with a focus on critical cues to exclude severe illness.
Conclusions
The findings support suggestions that improvements in primary care OOH could be achieved by including automated and regular timely feedback system for GPs and individual peer and expert clinician support for GPs with regular meetings to discuss recent cases. In addition, trainee support and mentoring to focus on clinical skills, knowledge and risk management issues specific to OOH is currently required. Investigating the stopping rules used for diagnostic closure may provide new insights into the root causes of clinical error in such a high-risk setting.
Article summary
Article focus
Clinical reasoning and decision making in an out of hours (OOH) primary care setting.
The aim is to gain insights into how general practitioners (GPs) make clinical decisions and manage risk in this environment.
Implications for system changes and training.
Key messages
Clinical decision making in OOH is dominated by rule-out strategies for severe illness or potentially high-risk diseases.
GPs use three main criteria to determine diagnostic closure: global wellness with rule-outs, responded to patient needs, presence of a reliable safety net.
Improvements to clinical decision making could be achieved by providing routine feedback to clinical staff working in OOH, building in systems to support reflection on clinical cases and more tailored GP training.
Strengths and limitations of this study
The design of the study is based on a strong theoretical framework provided by the dual theory of cognition.
Face validity through using recently seen cases.
Limitations relate primarily to sampling, participants consisting of self-selected individuals.
doi:10.1136/bmjopen-2011-000414
PMCID: PMC3330259  PMID: 22318661
3.  The causes of prescribing errors in English general practices: a qualitative study 
The British Journal of General Practice  2013;63(615):e713-e720.
Background
Few detailed studies exist of the underlying causes of prescribing errors in the UK.
Aim
To examine the causes of prescribing and monitoring errors in general practice and provide recommendations for how they may be overcome.
Design and setting
Qualitative interview and focus group study with purposive sampling of English general practices.
Method
General practice staff from 15 general practices across three PCTs in England participated in a combination of semi-structured interviews (n = 34) and six focus groups (n = 46). Thematic analysis informed by Reason’s Accident Causation Model was used.
Results
Seven categories of high-level error-producing conditions were identified: the prescriber, the patient, the team, the working environment, the task, the computer system, and the primary–secondary care interface. These were broken down to reveal various error-producing conditions: the prescriber’s therapeutic training, drug knowledge and experience, knowledge of the patient, perception of risk, and their physical and emotional health; the patient’s characteristics and the complexity of the individual clinical case; the importance of feeling comfortable within the practice team was highlighted, as well as the safety implications of GPs signing prescriptions generated by nurses when they had not seen the patient for themselves; the working environment with its extensive workload, time pressures, and interruptions; and computer-related issues associated with mis-selecting drugs from electronic pick-lists and overriding alerts were all highlighted as possible causes of prescribing errors and were often interconnected.
Conclusion
Complex underlying causes of prescribing and monitoring errors in general practices were highlighted, several of which are amenable to intervention.
doi:10.3399/bjgp13X673739
PMCID: PMC3782805  PMID: 24152487
general practice; medication safety; patient safety; prescribing; primary care; quality
4.  Effects of Two Commercial Electronic Prescribing Systems on Prescribing Error Rates in Hospital In-Patients: A Before and After Study 
PLoS Medicine  2012;9(1):e1001164.
In a before-and-after study, Johanna Westbrook and colleagues evaluate the change in prescribing error rates after the introduction of two commercial electronic prescribing systems in two Australian hospitals.
Background
Considerable investments are being made in commercial electronic prescribing systems (e-prescribing) in many countries. Few studies have measured or evaluated their effectiveness at reducing prescribing error rates, and interactions between system design and errors are not well understood, despite increasing concerns regarding new errors associated with system use. This study evaluated the effectiveness of two commercial e-prescribing systems in reducing prescribing error rates and their propensities for introducing new types of error.
Methods and Results
We conducted a before and after study involving medication chart audit of 3,291 admissions (1,923 at baseline and 1,368 post e-prescribing system) at two Australian teaching hospitals. In Hospital A, the Cerner Millennium e-prescribing system was implemented on one ward, and three wards, which did not receive the e-prescribing system, acted as controls. In Hospital B, the iSoft MedChart system was implemented on two wards and we compared before and after error rates. Procedural (e.g., unclear and incomplete prescribing orders) and clinical (e.g., wrong dose, wrong drug) errors were identified. Prescribing error rates per admission and per 100 patient days; rates of serious errors (5-point severity scale, those ≥3 were categorised as serious) by hospital and study period; and rates and categories of postintervention “system-related” errors (where system functionality or design contributed to the error) were calculated. Use of an e-prescribing system was associated with a statistically significant reduction in error rates in all three intervention wards (respectively reductions of 66.1% [95% CI 53.9%–78.3%]; 57.5% [33.8%–81.2%]; and 60.5% [48.5%–72.4%]). The use of the system resulted in a decline in errors at Hospital A from 6.25 per admission (95% CI 5.23–7.28) to 2.12 (95% CI 1.71–2.54; p<0.0001) and at Hospital B from 3.62 (95% CI 3.30–3.93) to 1.46 (95% CI 1.20–1.73; p<0.0001). This decrease was driven by a large reduction in unclear, illegal, and incomplete orders. The Hospital A control wards experienced no significant change (respectively −12.8% [95% CI −41.1% to 15.5%]; −11.3% [−40.1% to 17.5%]; −20.1% [−52.2% to 12.4%]). There was limited change in clinical error rates, but serious errors decreased by 44% (0.25 per admission to 0.14; p = 0.0002) across the intervention wards compared to the control wards (17% reduction; 0.30–0.25; p = 0.40). Both hospitals experienced system-related errors (0.73 and 0.51 per admission), which accounted for 35% of postsystem errors in the intervention wards; each system was associated with different types of system-related errors.
Conclusions
Implementation of these commercial e-prescribing systems resulted in statistically significant reductions in prescribing error rates. Reductions in clinical errors were limited in the absence of substantial decision support, but a statistically significant decline in serious errors was observed. System-related errors require close attention as they are frequent, but are potentially remediable by system redesign and user training. Limitations included a lack of control wards at Hospital B and an inability to randomize wards to the intervention.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Medication errors—for example, prescribing the wrong drug or giving a drug by the wrong route—frequently occur in health care settings and are responsible for thousands of deaths every year. Until recently, medicines were prescribed and dispensed using systems based on hand-written scripts. In hospitals, for example, physicians wrote orders for medications directly onto a medication chart, which was then used by the nursing staff to give drugs to their patients. However, drugs are now increasingly being prescribed using electronic prescribing (e-prescribing) systems. With these systems, prescribers use a computer and order medications for their patients with the help of a drug information database and menu items, free text boxes, and prewritten orders for specific conditions (so-called passive decision support). The system reviews the patient's medication and known allergy list and alerts the physician to any potential problems, including drug interactions (active decision support). Then after the physician has responded to these alerts, the order is transmitted electronically to the pharmacy and/or the nursing staff who administer the prescription.
Why Was This Study Done?
By avoiding the need for physicians to write out prescriptions and by providing active and passive decision support, e-prescribing has the potential to reduce medication errors. But, even though many countries are investing in expensive commercial e-prescribing systems, few studies have evaluated the effects of these systems on prescribing error rates. Moreover, little is known about the interactions between system design and errors despite fears that e-prescribing might introduce new errors. In this study, the researchers analyze prescribing error rates in hospital in-patients before and after the implementation of two commercial e-prescribing systems.
What Did the Researchers Do and Find?
The researchers examined medication charts for procedural errors (unclear, incomplete, or illegal orders) and for clinical errors (for example, wrong drug or dose) at two Australian hospitals before and after the introduction of commercial e-prescribing systems. At Hospital A, the Cerner Millennium e-prescribing system was introduced on one ward; three other wards acted as controls. At Hospital B, the researchers compared the error rates on two wards before and after the introduction of the iSoft MedChart e-prescribing system. The introduction of an e-prescribing system was associated with a substantial reduction in error rates in the three intervention wards; error rates on the control wards did not change significantly during the study. At Hospital A, medication errors declined from 6.25 to 2.12 per admission after the introduction of e-prescribing whereas at Hospital B, they declined from 3.62 to 1.46 per admission. This reduction in error rates was mainly driven by a reduction in procedural error rates and there was only a limited change in overall clinical error rates. Notably, however, the rate of serious errors decreased across the intervention wards from 0.25 to 0.14 per admission (a 44% reduction), whereas the serious error rate only decreased by 17% in the control wards during the study. Finally, system-related errors (for example, selection of an inappropriate drug located on a drop-down menu next to a likely drug selection) accounted for 35% of errors in the intervention wards after the implementation of e-prescribing.
What Do These Findings Mean?
These findings show that the implementation of these two e-prescribing systems markedly reduced hospital in-patient prescribing error rates, mainly by reducing the number of incomplete, illegal, or unclear medication orders. The limited decision support built into both the e-prescribing systems used here may explain the limited reduction in clinical error rates but, importantly, both e-prescribing systems reduced serious medication errors. Finally, the high rate of system-related errors recorded in this study is worrying but is potentially remediable by system redesign and user training. Because this was a “real-world” study, it was not possible to choose the intervention wards randomly. Moreover, there was no control ward at Hospital B, and the wards included in the study had very different specialties. These and other aspects of the study design may limit the generalizability of these findings, which need to be confirmed and extended in additional studies. Even so, these findings provide persuasive evidence of the current and potential ability of commercial e-prescribing systems to reduce prescribing errors in hospital in-patients provided these systems are continually monitored and refined to improve their performance.
Additional Information
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001164.
ClinfoWiki has pages on medication errors and on electronic prescribing (note: the Clinical Informatics Wiki is a free online resource that anyone can add to or edit)
Electronic prescribing in hospitals challenges and lessons learned describes the implementation of e-prescribing in UK hospitals; more information about e-prescribing in the UK is available on the NHS Connecting for Health Website
The Clinicians Guide to e-Prescribing provides up-to-date information about e-prescribing in the USA
Information about e-prescribing in Australia is also available
Information about electronic health records in Australia
doi:10.1371/journal.pmed.1001164
PMCID: PMC3269428  PMID: 22303286
5.  Identification of factors associated with diagnostic error in primary care 
BMC Family Practice  2014;15:92.
Background
Missed, delayed or incorrect diagnoses are considered to be diagnostic errors. The aim of this paper is to describe the methodology of a study to analyse cognitive aspects of the process by which primary care (PC) physicians diagnose dyspnoea. It examines the possible links between the use of heuristics, suboptimal cognitive acts and diagnostic errors, using Reason’s taxonomy of human error (slips, lapses, mistakes and violations). The influence of situational factors (professional experience, perceived overwork and fatigue) is also analysed.
Methods
Cohort study of new episodes of dyspnoea in patients receiving care from family physicians and residents at PC centres in Granada (Spain). With an initial expected diagnostic error rate of 20%, and a sampling error of 3%, 384 episodes of dyspnoea are calculated to be required. In addition to filling out the electronic medical record of the patients attended, each physician fills out 2 specially designed questionnaires about the diagnostic process performed in each case of dyspnoea. The first questionnaire includes questions on the physician’s initial diagnostic impression, the 3 most likely diagnoses (in order of likelihood), and the diagnosis reached after the initial medical history and physical examination. It also includes items on the physicians’ perceived overwork and fatigue during patient care. The second questionnaire records the confirmed diagnosis once it is reached. The complete diagnostic process is peer-reviewed to identify and classify the diagnostic errors. The possible use of heuristics of representativeness, availability, and anchoring and adjustment in each diagnostic process is also analysed. Each audit is reviewed with the physician responsible for the diagnostic process. Finally, logistic regression models are used to determine if there are differences in the diagnostic error variables based on the heuristics identified.
Discussion
This work sets out a new approach to studying the diagnostic decision-making process in PC, taking advantage of new technologies which allow immediate recording of the decision-making process.
doi:10.1186/1471-2296-15-92
PMCID: PMC4024115  PMID: 24884984
Primary care; Diagnostic errors; Decision-making
6.  Patient-Safety-Related Hospital Deaths in England: Thematic Analysis of Incidents Reported to a National Database, 2010–2012 
PLoS Medicine  2014;11(6):e1001667.
Sukhmeet Panesar and colleagues classified reports of patient-safety-related hospital deaths in England to identify patterns of cases where improvements might be possible.
Please see later in the article for the Editors' Summary
Background
Hospital mortality is increasingly being regarded as a key indicator of patient safety, yet methodologies for assessing mortality are frequently contested and seldom point directly to areas of risk and solutions. The aim of our study was to classify reports of deaths due to unsafe care into broad areas of systemic failure capable of being addressed by stronger policies, procedures, and practices. The deaths were reported to a patient safety incident reporting system after mandatory reporting of such incidents was introduced.
Methods and Findings
The UK National Health Service database was searched for incidents resulting in a reported death of an adult over the period of the study. The study population comprised 2,010 incidents involving patients aged 16 y and over in acute hospital settings. Each incident report was reviewed by two of the authors, and, by scrutinising the structured information together with the free text, a main reason for the harm was identified and recorded as one of 18 incident types. These incident types were then aggregated into six areas of apparent systemic failure: mismanagement of deterioration (35%), failure of prevention (26%), deficient checking and oversight (11%), dysfunctional patient flow (10%), equipment-related errors (6%), and other (12%). The most common incident types were failure to act on or recognise deterioration (23%), inpatient falls (10%), healthcare-associated infections (10%), unexpected per-operative death (6%), and poor or inadequate handover (5%). Analysis of these 2,010 fatal incidents reveals patterns of issues that point to actionable areas for improvement.
Conclusions
Our approach demonstrates the potential utility of patient safety incident reports in identifying areas of service failure and highlights opportunities for corrective action to save lives.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Being admitted to the hospital is worrying for patients and for their relatives. Will the patient recover or die in the hospital? Some seriously ill patients will inevitably die, but in an ideal world, no one should die in the hospital because of inadequate or unsafe care (an avoidable death). No one should die, for example, because healthcare professionals fail to act on signs that indicate a decline in a patient's clinical condition. Hospital mortality (death) is often regarded as a key indicator of patient safety in hospitals, and death rate indicators such as the “hospital standardized mortality ratio” (the ratio of the actual number of acute in-hospital deaths to the expected number of in-hospital deaths) are widely used to monitor and improve hospital safety standards. In England, for example, a 2012 report that included this measure as an indicator of hospital performance led to headlines of “worryingly high” hospital death rates and to a review of the quality of care in the hospitals with the highest death rates.
Why Was This Study Done?
Hospital standardized mortality ratios and other measures of in-patient mortality can be misleading because they can, for example, reflect the burden of disease near the hospital rather than the hospital's quality of care or safety levels. Moreover, comparative data on hospital mortality rates are of limited value in identifying areas of risk to patients or solutions to the problem of avoidable deaths. In this study, to identify areas of service failure amenable to improvement through strengthened clinical policies, procedures, and practices, the researchers undertake a thematic analysis of deaths in hospitals in England that were reported by healthcare staff to a mandatory patient-safety-related incident reporting system. Since 2004, staff in the UK National Health Service (the NHS comprises the publicly funded healthcare systems in England, Scotland, Wales, and Northern Ireland) have been encouraged to report any unintended or unexpected incident in which they believe a patient's safety was compromised. Since June 2010, it has been mandatory for staff in England and Wales to report deaths due to patient-safety-related incidents. A thematic analysis examines patterns (“themes”) within nonnumerical (qualitative) data.
What Did the Researchers Do and Find?
By searching the NHS database of patient-safety-related incidents, the researchers identified 2010 incidents that occurred between 1 June 2010 and 31 October 2012 that resulted in the death of adult patients in acute hospital settings. By scrutinizing the structured information in each incident report and the associated free text in which the reporter described what happened and why they think it happened, the researchers classified the reports into 18 incident categories. These categories fell into six broad areas of systemic failure—mismanagement of deterioration (35% of incidents), failure of prevention (26%), deficient checking and oversight (11%), dysfunctional patient flow (10%), equipment-related errors (6%), and other (12%, incidents where the problem underlying death was unclear). Management of deterioration, for example, included the incident categories “failure to act on or recognize deterioration” (23% of reported incidents), “failure to give ordered treatment/support in a timely manner,” and “failure to observe.” Failure of prevention included the incident categories “falls” (10% of reported incidents), “healthcare-associated infections” (also 10% of reported incidents), “pressure sores,” “suicides,” and “deep vein thrombosis/pulmonary embolism.”
What Do These Findings Mean?
Although the accuracy of these findings may be limited by data quality and by other aspects of the study design, they reveal patterns of patient-safety-related deaths in hospitals in England and highlight areas of healthcare that can be targeted for improvement. The finding that the mismanagement of deterioration of acutely ill patients is involved in a third of patient-safety-related deaths identifies an area of particular concern in the NHS and, potentially, in other healthcare systems. One way to reduce deaths associated with the mismanagement of deterioration, suggest the researchers, might be to introduce a standardized early warning score to ensure uniform identification of this population of patients. The researchers also suggest that more effort should be put into designing programs to prevent falls and other incidents and into ensuring that these programs are effectively implemented. More generally, the classification system developed here has the potential to help hospital boards and clinicians identify areas of patient care that require greater scrutiny and intervention and thereby save the lives of many hospital patients.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001667.
The NHS provides information about patient safety, including a definition of a patient safety incident and information on reporting patient safety incidents
The NHS Choices website includes several “Behind the Headlines” articles that discuss patient safety in hospitals, including an article that discusses the 2012 report of high hospital death rates in England, “Fit for the Future?” and another that discusses the Keogh review of the quality of care in the hospitals with highest death rates
The US Agency for Healthcare Research and Quality provides information on patient safety in the US
Wikipedia has pages on thematic analysis and on patient safety (note that Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
doi:10.1371/journal.pmed.1001667
PMCID: PMC4068985  PMID: 24959751
7.  Dengue Contingency Planning: From Research to Policy and Practice 
PLoS Neglected Tropical Diseases  2016;10(9):e0004916.
Background
Dengue is an increasingly incident disease across many parts of the world. In response, an evidence-based handbook to translate research into policy and practice was developed. This handbook facilitates contingency planning as well as the development and use of early warning and response systems for dengue fever epidemics, by identifying decision-making processes that contribute to the success or failure of dengue surveillance, as well as triggers that initiate effective responses to incipient outbreaks.
Methodology/Principal findings
Available evidence was evaluated using a step-wise process that included systematic literature reviews, policymaker and stakeholder interviews, a study to assess dengue contingency planning and outbreak management in 10 countries, and a retrospective logistic regression analysis to identify alarm signals for an outbreak warning system using datasets from five dengue endemic countries. Best practices for managing a dengue outbreak are provided for key elements of a dengue contingency plan including timely contingency planning, the importance of a detailed, context-specific dengue contingency plan that clearly distinguishes between routine and outbreak interventions, surveillance systems for outbreak preparedness, outbreak definitions, alert algorithms, managerial capacity, vector control capacity, and clinical management of large caseloads. Additionally, a computer-assisted early warning system, which enables countries to identify and respond to context-specific variables that predict forthcoming dengue outbreaks, has been developed.
Conclusions/Significance
Most countries do not have comprehensive, detailed contingency plans for dengue outbreaks. Countries tend to rely on intensified vector control as their outbreak response, with minimal focus on integrated management of clinical care, epidemiological, laboratory and vector surveillance, and risk communication. The Technical Handbook for Surveillance, Dengue Outbreak Prediction/ Detection and Outbreak Response seeks to provide countries with evidence-based best practices to justify the declaration of an outbreak and the mobilization of the resources required to implement an effective dengue contingency plan.
Author Summary
An evidence-based handbook was generated to facilitate deployment of dengue surveillance and response systems for timely and effective management of outbreaks, and to identify the factors required for success. Evidence was evaluated using literature reviews, policymaker and stakeholder interviews, assessment of dengue contingency planning and outbreak management in ten endemic countries, and a statistical analysis to identify outbreak early warning signs in five countries. Best practices for managing dengue outbreaks included timely and context-specific dengue contingency plans that distinguished between routine practices and outbreak interventions, surveillance systems, outbreak definitions, alert algorithms, and managerial, clinical and vector control capacity. A computer-assisted early warning system was developed to enable each locality to develop its own context-specific scheme. Today, most countries do not have comprehensive, detailed contingency plans for dengue outbreaks, responding simply by intensifying vector control, with minimal focus on integrated management of clinical care, epidemiological, laboratory and vector surveillance, and risk communication. To rectify this, our handbook provides countries with evidence-based best practices to justify the declaration of an outbreak and for the mobilization and management of appropriate resources required to implement a dengue contingency plan.
doi:10.1371/journal.pntd.0004916
PMCID: PMC5031449  PMID: 27653786
8.  Evaluation of the effects of implementing an electronic early warning score system: protocol for a stepped wedge study 
Background
An Early Warning Score is a clinical risk score based upon vital signs intended to aid recognition of patients in need of urgent medical attention. The use of an escalation of care policy based upon an Early Warning Score is mandated as the standard of practice in British hospitals. Electronic systems for recording vital sign observations and Early Warning Score calculation offer theoretical benefits over paper-based systems. However, the evidence for their clinical benefit is limited. Previous studies have shown inconsistent results. The majority have employed a “before and after” study design, which may be strongly confounded by simultaneously occurring events. This study aims to examine how the implementation of an electronic early warning score system, System for Notification and Documentation (SEND), affects the recognition of clinical deterioration occurring in hospitalised adult patients.
Methods
This study is a non-randomised stepped wedge evaluation carried out across the four hospitals of the Oxford University Hospitals NHS Trust, comparing charting on paper and charting using SEND. We assume that more frequent monitoring of acutely ill patients is associated with better recognition of patient deterioration.
The primary outcome measure is the time between a patient’s first observations set with an Early Warning Score above the alerting threshold and their subsequent set of observations. Secondary outcome measures are in-hospital mortality, cardiac arrest and Intensive Care admission rates, hospital length of stay and system usability measured using the System Usability Scale. We will also measure Intensive Care length of stay, Intensive Care mortality, Acute Physiology and Chronic Health Evaluation (APACHE) II acute physiology score on admission, to examine whether the introduction of SEND has any effect on Intensive Care-related outcomes.
Discussion
The development of this protocol has been informed by guidance from the Agency for Healthcare Research and Quality (AHRQ) Health Information Technology Evaluation Toolkit and Delone and McLeans’s Model of Information System Success. Our chosen trial design, a stepped wedge study, is well suited to the study of a phased roll out. The choice of primary endpoint is challenging. We have selected the time from the first triggering observation set to the subsequent observation set. This has the benefit of being easy to measure on both paper and electronic charting and having a straightforward interpretation. We have collected qualitative measures of system quality via a user questionnaire and organisational descriptors to help readers understand the context in which SEND has been implemented.
Electronic supplementary material
The online version of this article (doi:10.1186/s12911-016-0257-8) contains supplementary material, which is available to authorized users.
doi:10.1186/s12911-016-0257-8
PMCID: PMC4748571  PMID: 26860362
Vital Signs; Early Warning Score; Track and Trigger; Electronic Charting; Stepped-Wedge
9.  Missed opportunities for diagnosis: lessons learned from diagnostic errors in primary care 
The British Journal of General Practice  2015;65(641):e838-e844.
Background
Because of the difficulties inherent in diagnosis in primary care, it is inevitable that diagnostic errors will occur. However, despite the important consequences associated with diagnostic errors and their estimated high prevalence, teaching and research on diagnostic error is a neglected area.
Aim
To ascertain the key learning points from GPs’ experiences of diagnostic errors and approaches to clinical decision making associated with these.
Design and setting
Secondary analysis of 36 qualitative interviews with GPs in Oxfordshire, UK.
Method
Two datasets of semi-structured interviews were combined. Questions focused on GPs’ experiences of diagnosis and diagnostic errors (or near misses) in routine primary care and out of hours. Interviews were audiorecorded, transcribed verbatim, and analysed thematically.
Results
Learning points include GPs’ reliance on ‘pattern recognition’ and the failure of this strategy to identify atypical presentations; the importance of considering all potentially serious conditions using a ‘restricted rule out’ approach; and identifying and acting on a sense of unease. Strategies to help manage uncertainty in primary care were also discussed.
Conclusion
Learning from previous examples of diagnostic errors is essential if these events are to be reduced in the future and this should be incorporated into GP training. At a practice level, learning points from experiences of diagnostic errors should be discussed more frequently; and more should be done to integrate these lessons nationally to understand and characterise diagnostic errors.
doi:10.3399/bjgp15X687889
PMCID: PMC4655738  PMID: 26622037
clinical reasoning; decision making; diagnosis; diagnostic errors; education; general practice
10.  The Effect of Automated Alerts on Provider Ordering Behavior in an Outpatient Setting 
PLoS Medicine  2005;2(9):e255.
Background
Computerized order entry systems have the potential to prevent medication errors and decrease adverse drug events with the use of clinical-decision support systems presenting alerts to providers. Despite the large volume of medications prescribed in the outpatient setting, few studies have assessed the impact of automated alerts on medication errors related to drug–laboratory interactions in an outpatient primary-care setting.
Methods and Findings
A primary-care clinic in an integrated safety net institution was the setting for the study. In collaboration with commercial information technology vendors, rules were developed to address a set of drug–laboratory interactions. All patients seen in the clinic during the study period were eligible for the intervention. As providers ordered medications on a computer, an alert was displayed if a relevant drug–laboratory interaction existed. Comparisons were made between baseline and postintervention time periods. Provider ordering behavior was monitored focusing on the number of medication orders not completed and the number of rule-associated laboratory test orders initiated after alert display. Adverse drug events were assessed by doing a random sample of chart reviews using the Naranjo scoring scale.
The rule processed 16,291 times during the study period on all possible medication orders: 7,017 during the pre-intervention period and 9,274 during the postintervention period. During the postintervention period, an alert was displayed for 11.8% (1,093 out of 9,274) of the times the rule processed, with 5.6% for only “missing laboratory values,” 6.0% for only “abnormal laboratory values,” and 0.2% for both types of alerts. Focusing on 18 high-volume and high-risk medications revealed a significant increase in the percentage of time the provider stopped the ordering process and did not complete the medication order when an alert for an abnormal rule-associated laboratory result was displayed (5.6% vs. 10.9%, p = 0.03, Generalized Estimating Equations test). The provider also increased ordering of the rule-associated laboratory test when an alert was displayed (39% at baseline vs. 51% during post intervention, p < 0.001). There was a non-statistically significant difference towards less “definite” or “probable” adverse drug events defined by Naranjo scoring (10.3% at baseline vs. 4.3% during postintervention, p = 0.23).
Conclusion
Providers will adhere to alerts and will use this information to improve patient care. Specifically, in response to drug–laboratory interaction alerts, providers will significantly increase the ordering of appropriate laboratory tests. There may be a concomitant change in adverse drug events that would require a larger study to confirm. Implementation of rules technology to prevent medication errors could be an effective tool for reducing medication errors in an outpatient setting.
A computerized order entry system that alerted providers to potential problems was shown to be able to influence prescribing practice
doi:10.1371/journal.pmed.0020255
PMCID: PMC1198038  PMID: 16128621
11.  Can an electronic prescribing system detect doctors who are more likely to make a serious prescribing error? 
Objectives
We aimed to assess whether routine data produced by an electronic prescribing system might be useful in identifying doctors at higher risk of making a serious prescribing error.
Design
Retrospective analysis of prescribing by junior doctors over 12 months using an electronic prescribing information and communication system. The system issues a graded series of prescribing alerts (low-level, intermediate, and high-level), and warnings and prompts to respond to abnormal test results. These may be overridden or heeded, except for high-level prescribing alerts, which are indicative of a potentially serious error and impose a ‘hard stop’.
Setting
A large teaching hospital.
Participants
All junior doctors in the study setting.
Main outcome measures
Rates of prescribing alerts and laboratory warnings and doctors' responses.
Results
Altogether 848,678 completed prescriptions issued by 381 doctors (median 1538 prescriptions per doctor, interquartile range [IQR] 328–3275) were analysed. We identified 895,029 low-level alerts (median 1033 per 1000 prescriptions per doctor, IQR 903–1205) with a median of 34% (IQR 31–39%) heeded; 172,434 intermediate alerts (median 196 per 1000 prescriptions per doctor, IQR 159–266), with a median of 23% (IQR 16–30%) heeded; and 11,940 high-level ‘hard stop’ alerts. Doctors vary greatly in the extent to which they trigger and respond to alerts of different types. The rate of high-level alerts showed weak correlation with the rate of intermediate prescribing alerts (correlation coefficient, r = 0.40, P = <0.001); very weak correlation with low-level alerts (r = 0.12, P = 0.019); and showed weak (and sometimes negative) correlation with propensity to heed test-related warnings or alarms. The degree of correlation between generation of intermediate and high-level alerts is insufficient to identify doctors at high risk of making serious errors.
Conclusions
Routine data from an electronic prescribing system should not be used to identify doctors who are at risk of making serious errors. Careful evaluation of the kinds of quality assurance questions for which routine data are suitable will be increasingly valuable.
doi:10.1258/jrsm.2011.110061
PMCID: PMC3089874  PMID: 21558099
12.  Diagnostic Errors in Pediatric Echocardiography Development of Taxonomy and Identification of Risk Factors 
Circulation  2008;117(23):2995-3001.
Background
Despite increased interest in complications within pediatric cardiology, the domain of imaging-related diagnostic errors has received little attention. We developed a new taxonomy for diagnostic errors within pediatric echocardiography that categorizes errors by severity, preventability, and primary contributor. Our objectives were to examine its findings when applied to diagnostic error cases and to identify risk factors for preventable or possibly preventable diagnostic errors.
Methods and Results
Diagnostic errors were identified at a high-volume academic pediatric cardiac center from December 2004 to August 2007. Demographic, clinical, and situational variables were collected from these cases and controls. During the study period, ≈50 660 echocardiograms were performed. Among the 87 diagnostic error cases identified, 70% affected clinical management or the patient was at risk of or experienced an adverse event. One third of the errors were preventable and 46% were possibly preventable; 69% of preventable errors were of moderate severity or greater. Univariate analysis demonstrated that preventable or possibly preventable errors were more likely to involve younger patients, lower body weight, study location, sedated/anesthetized patients, studies performed and interpreted at night, uncommon diagnoses, and greater anatomic complexity than controls. Multivariate analysis identified the following risk factors: rare or very rare diagnoses (adjusted odds ratio [AOR], 9.2; P<0.001), study location in the recovery room (AOR, 7.9; P<0.001), moderate anatomic complexity (AOR, 3.5; P=0.004), and patient weight <5 kg (AOR, 3.5; P=0.031).
Conclusions
A diagnostic error taxonomy and knowledge of risk factors can assist in identification of targets for quality improvement initiatives that aim to decrease diagnostic error in pediatric echocardiography.
doi:10.1161/CIRCULATIONAHA.107.758532
PMCID: PMC4237021  PMID: 18519849
diagnostic errors; echocardiography; congenital heart disease; pediatrics
13.  Cognitive biases associated with medical decisions: a systematic review 
Background
Cognitive biases and personality traits (aversion to risk or ambiguity) may lead to diagnostic inaccuracies and medical errors resulting in mismanagement or inadequate utilization of resources. We conducted a systematic review with four objectives: 1) to identify the most common cognitive biases, 2) to evaluate the influence of cognitive biases on diagnostic accuracy or management errors, 3) to determine their impact on patient outcomes, and 4) to identify literature gaps.
Methods
We searched MEDLINE and the Cochrane Library databases for relevant articles on cognitive biases from 1980 to May 2015. We included studies conducted in physicians that evaluated at least one cognitive factor using case-vignettes or real scenarios and reported an associated outcome written in English. Data quality was assessed by the Newcastle-Ottawa scale. Among 114 publications, 20 studies comprising 6810 physicians met the inclusion criteria. Nineteen cognitive biases were identified.
Results
All studies found at least one cognitive bias or personality trait to affect physicians. Overconfidence, lower tolerance to risk, the anchoring effect, and information and availability biases were associated with diagnostic inaccuracies in 36.5 to 77 % of case-scenarios. Five out of seven (71.4 %) studies showed an association between cognitive biases and therapeutic or management errors. Of two (10 %) studies evaluating the impact of cognitive biases or personality traits on patient outcomes, only one showed that higher tolerance to ambiguity was associated with increased medical complications (9.7 % vs 6.5 %; p = .004). Most studies (60 %) targeted cognitive biases in diagnostic tasks, fewer focused on treatment or management (35 %) and on prognosis (10 %). Literature gaps include potentially relevant biases (e.g. aggregate bias, feedback sanction, hindsight bias) not investigated in the included studies. Moreover, only five (25 %) studies used clinical guidelines as the framework to determine diagnostic or treatment errors. Most studies (n = 12, 60 %) were classified as low quality.
Conclusions
Overconfidence, the anchoring effect, information and availability bias, and tolerance to risk may be associated with diagnostic inaccuracies or suboptimal management. More comprehensive studies are needed to determine the prevalence of cognitive biases and personality traits and their potential impact on physicians’ decisions, medical errors, and patient outcomes.
Electronic supplementary material
The online version of this article (doi:10.1186/s12911-016-0377-1) contains supplementary material, which is available to authorized users.
doi:10.1186/s12911-016-0377-1
PMCID: PMC5093937  PMID: 27809908
Decision making; Cognitive bias; Personality traits; Cognition; Physicians; Case-scenarios; Systematic review
14.  A Molecular Host Response Assay to Discriminate Between Sepsis and Infection-Negative Systemic Inflammation in Critically Ill Patients: Discovery and Validation in Independent Cohorts 
PLoS Medicine  2015;12(12):e1001916.
Background
Systemic inflammation is a whole body reaction having an infection-positive (i.e., sepsis) or infection-negative origin. It is important to distinguish between these two etiologies early and accurately because this has significant therapeutic implications for critically ill patients. We hypothesized that a molecular classifier based on peripheral blood RNAs could be discovered that would (1) determine which patients with systemic inflammation had sepsis, (2) be robust across independent patient cohorts, (3) be insensitive to disease severity, and (4) provide diagnostic utility. The goal of this study was to identify and validate such a molecular classifier.
Methods and Findings
We conducted an observational, non-interventional study of adult patients recruited from tertiary intensive care units (ICUs). Biomarker discovery utilized an Australian cohort (n = 105) consisting of 74 cases (sepsis patients) and 31 controls (post-surgical patients with infection-negative systemic inflammation) recruited at five tertiary care settings in Brisbane, Australia, from June 3, 2008, to December 22, 2011. A four-gene classifier combining CEACAM4, LAMP1, PLA2G7, and PLAC8 RNA biomarkers was identified. This classifier, designated SeptiCyte Lab, was validated using reverse transcription quantitative PCR and receiver operating characteristic (ROC) curve analysis in five cohorts (n = 345) from the Netherlands. Patients for validation were selected from the Molecular Diagnosis and Risk Stratification of Sepsis study (ClinicalTrials.gov, NCT01905033), which recruited ICU patients from the Academic Medical Center in Amsterdam and the University Medical Center Utrecht. Patients recruited from November 30, 2012, to August 5, 2013, were eligible for inclusion in the present study. Validation cohort 1 (n = 59) consisted entirely of unambiguous cases and controls; SeptiCyte Lab gave an area under curve (AUC) of 0.95 (95% CI 0.91–1.00) in this cohort. ROC curve analysis of an independent, more heterogeneous group of patients (validation cohorts 2–5; 249 patients after excluding 37 patients with an infection likelihood of “possible”) gave an AUC of 0.89 (95% CI 0.85–0.93). Disease severity, as measured by Sequential Organ Failure Assessment (SOFA) score or Acute Physiology and Chronic Health Evaluation (APACHE) IV score, was not a significant confounding variable. The diagnostic utility of SeptiCyte Lab was evaluated by comparison to various clinical and laboratory parameters available to a clinician within 24 h of ICU admission. SeptiCyte Lab was significantly better at differentiating cases from controls than all tested parameters, both singly and in various logistic combinations, and more than halved the diagnostic error rate compared to procalcitonin in all tested cohorts and cohort combinations. Limitations of this study relate to (1) cohort compositions that do not perfectly reflect the composition of the intended use population, (2) potential biases that could be introduced as a result of the current lack of a gold standard for diagnosing sepsis, and (3) lack of a complete, unbiased comparison to C-reactive protein.
Conclusions
SeptiCyte Lab is a rapid molecular assay that may be clinically useful in managing ICU patients with systemic inflammation. Further study in population-based cohorts is needed to validate this assay for clinical use.
Thomas Yager and colleagues develop and validate a molecular host assay to discriminate between sepsis and infection-negative systemic inflammation in critically ill patients.
Editors' Summary
Background
Our immune system protects us from disease by recognizing and killing bacteria and other infectious organisms (pathogens). An important part of the immune response is inflammation, a process that is triggered by infection, tissue injury, and other stimuli. Localized inflammation, which is characterized by swelling, redness, heat, pain, and loss of function, restricts the tissue damage caused by these stimuli to the affected site. Unfortunately, the immune system occasionally initiates a series of reactions that lead to widespread (systemic) inflammation. Systemic inflammation can damage vital organs and can be life-threatening. For example, it has been estimated that 30%–50% of people who develop severe infection-positive systemic inflammation (sepsis) die. Clinical management of systemic inflammation depends on whether the condition is caused by an infection or by another stimulus: many patients with systemic inflammation need to be given corticosteroids and other anti-inflammatory agents, but only patients with infection-positive systemic inflammation need to be given antibiotics.
Why Was This Study Done?
Clinicians need to be able to distinguish between sepsis and infection-negative systemic inflammation quickly and accurately when treating critically ill patients. Patients with sepsis need to be given antibiotics as soon as possible to clear the infection, but giving antibiotics to someone with infection-negative systemic inflammation may do more harm than good. Current diagnostic approaches for identifying patients with sepsis rely on isolating and identifying the causative pathogen, but it can take more than 24 hours to obtain a result and pathogens are only isolated from about 30% of patients with clinically confirmed sepsis. Analysis of the host immune response might provide a quicker, more accurate way to differentiate between sepsis and infection-negative systemic inflammation. Thus, measurement of blood levels of procalcitonin (a pro-inflammatory biomarker) or of C-reactive protein (which the liver releases in response to inflammation) can sometimes differentiate between the two conditions. However, because the immune response is complex, measurement of a single biomarker in the blood is unlikely to be diagnostically helpful in every patient. Here, the researchers identify and validate a set of RNA molecules present in the blood capable of discriminating between sepsis and infection-negative systemic inflammation in critically ill patients (a “molecular classifier”).
What Did the Researchers Do and Find?
Using microarray analysis (a technique that measures the RNA levels of thousands of different genes) to compare the RNA molecules present in the blood in a discovery cohort of 74 patients with sepsis and 31 post-surgical patients with infection-negative systemic inflammation, the researchers identified a molecular classifier (SeptiCyte Lab) consisting of four RNA biomarkers. They validated this classifier using five additional patient cohorts, RT-qPCR (a technique that measures the amounts of specific RNAs in biological samples), and receiver operating characteristic (ROC) curve analysis (a graphical method for determining diagnostic test performance). The overall AUC for the test in the five validation cohorts was 0.88. The AUC (area under a ROC curve) quantifies the ability of a test to discriminate between individuals with and without a disease. A perfect test that yields no false positives or false negatives has an AUC of 1.00; a test no better at identifying true positives than flipping a coin has an AUC of 0.5. Importantly, disease severity did not affect the performance of SeptiCyte Lab, and the assay was better at discriminating sepsis from infection-negative systemic inflammation than all tested clinical and laboratory parameters (singly and in combination) that can currently be obtained within 24 hours of admission to an intensive care unit.
What Do These Findings Mean?
These findings suggest that SeptiCyte Lab might be useful in the management of critically ill patients with systemic inflammation. In the validation cohorts tested, this rapid diagnostic test (SeptiCyte Lab currently takes 4–6 hours to perform but could be developed to provide a result in about 90 minutes) was able to correctly identify 90% of patients with sepsis, although it also mistakenly identified some patients as having sepsis when they had infection-negative systemic inflammation. Further clinical studies are needed before SeptiCyte Lab can be used clinically because of limitations in the current study. For example, the patients in the validation cohorts imperfectly reflect the composition of real-world patients with systemic inflammation. However, the researchers suggest that, in combination with other clinical parameters and clinical judgment, SeptiCyte Lab might help physicians make appropriate therapeutic decisions for patients with systemic inflammation.
Additional Information
This list of resources contains links that can be accessed when viewing the PDF on a device or via the online version of the article at http://dx.doi.org/10.1371/journal.pmed.1001916.
The UK National Health Service Choices website provides information on sepsis
The US National Institute of General Medical Sciences provides a fact sheet on sepsis
The UK Sepsis Trust is a not-for-profit organization that provides information and personal stories about sepsis
The World Sepsis Day website also provides information and personal stories about sepsis
MedlinePlus provides links to further information about sepsis (in English and Spanish)
Wikipedia has pages on sepsis, systemic inflammatory response syndrome, and ROC curve analysis (note that Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
doi:10.1371/journal.pmed.1001916
PMCID: PMC4672921  PMID: 26645559
15.  Stroke awareness in the general population: knowledge of stroke risk factors and warning signs in older adults 
BMC Geriatrics  2009;9:35.
Background
Stroke is a leading cause of death and functional impairment. While older people are particularly vulnerable to stroke, research suggests that they have the poorest awareness of stroke warning signs and risk factors. This study examined knowledge of stroke warning signs and risk factors among community-dwelling older adults.
Methods
Randomly selected community-dwelling older people (aged 65+) in Ireland (n = 2,033; 68% response rate). Participants completed home interviews. Questions assessed knowledge of stroke warning signs and risk factors, and personal risk factors for stroke.
Results
Of the overall sample, 6% had previously experienced a stroke or transient ischaemic attack. When asked to identify stroke risk factors from a provided list, less than half of the overall sample identified established risk factors (e.g., smoking, hypercholesterolaemia), hypertension being the only exception (identified by 74%). Similarly, less than half identified established warning signs (e.g., weakness, headache), with slurred speech (54%) as the exception. Overall, there were considerable gaps in awareness with poorest levels evident in those with primary level education only and in those living in Northern Ireland (compared with Republic of Ireland).
Conclusion
Knowledge deficits in this study suggest that most of the common early symptoms or signs of stroke were recognized as such by less than half of the older adults surveyed. As such, many older adults may not recognise early symptoms of stroke in themselves or others. Thus, they may lose vital time in presenting for medical attention. Lack of public awareness about stroke warning signs and risk factors must be addressed as one important contribution to reducing mortality and morbidity from stroke.
doi:10.1186/1471-2318-9-35
PMCID: PMC2734750  PMID: 19656359
16.  Types and Origins of Diagnostic Errors in Primary Care Settings 
JAMA internal medicine  2013;173(6):418-425.
Importance
Diagnostic errors are an understudied aspect of ambulatory patient safety.
Objectives
To determine the types of diseases missed and the diagnostic process involved in the cases of confirmed diagnosis errors in primary care settings and to determine whether record reviews could shed light on potential contributory factors to inform future interventions.
Design
We reviewed medical records of diagnostic errors detected at two sites through electronic health records-based triggers. Triggers were based on patterns of patients’ unexpected return visits after an initial primary care “index” visit.
Setting
A larger urban Veterans Affairs facility and a large integrated private health care system.
Participants
Our study focused on 190 unique instances of diagnostic errors detected in primary care visits between October 1, 2006, and September 30, 2007.
Main Outcome Measures
Through medical record reviews, we collected data on presenting symptoms at the index visit, types of diagnoses missed, process breakdowns, potential contributory factors, and potential for harm from errors.
Results
In 190 cases, a total of 68 unique diagnoses were missed. Most missed diagnoses were common conditions in primary care, with pneumonia (6.7%), decompensated congestive heart failure (5.7%), acute renal failure (5.3%), cancer (primary) (5.3%), and urinary tract infection or pyelonephritis (4.8%) being most common. Process breakdowns most frequently involved the patient-practitioner clinical encounter (78.9%) but were also related to referrals (19.5%), patient-related factors (16.3%), follow-up and tracking of diagnostic information (14.7%), and performance and interpretation of diagnostic tests (13.6%). A total of 43.7% of cases involved more than one of these processes. Patient- practitioner encounter breakdowns were primarily related to problems with history-taking (56.3%), examination (47.4%), and/or ordering diagnostic tests for further work-up (57.4%). Most errors were associated with potential for moderate-to-severe harm.
Conclusions and Relevance
Diagnostic errors identified in our study involved a large variety of common diseases and had significant potential for harm. Most errors were related to process breakdowns in the patient-practitioner clinical encounter. Preventive interventions should target common contributory factors across diagnoses, especially those that involve data gathering and synthesis in the patient- practitioner encounter.
doi:10.1001/jamainternmed.2013.2777
PMCID: PMC3690001  PMID: 23440149
diagnostic errors; patient safety; follow-up; primary care; electronic health records
17.  Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety 
Long working hours and sleep deprivation have been a facet of physician training in the US since the advent of the modern residency system. However, the scientific evidence linking fatigue with deficits in human performance, accidents and errors in industries from aeronautics to medicine, nuclear power, and transportation has mounted over the last 40 years. This evidence has also spawned regulations to help ensure public safety across safety-sensitive industries, with the notable exception of medicine.
In late 2007, at the behest of the US Congress, the Institute of Medicine embarked on a year-long examination of the scientific evidence linking resident physician sleep deprivation with clinical performance deficits and medical errors. The Institute of Medicine’s report, entitled “Resident duty hours: Enhancing sleep, supervision and safety”, published in January 2009, recommended new limits on resident physician work hours and workload, increased supervision, a heightened focus on resident physician safety, training in structured handovers and quality improvement, more rigorous external oversight of work hours and other aspects of residency training, and the identification of expanded funding sources necessary to implement the recommended reforms successfully and protect the public and resident physicians themselves from preventable harm.
Given that resident physicians comprise almost a quarter of all physicians who work in hospitals, and that taxpayers, through Medicare and Medicaid, fund graduate medical education, the public has a deep investment in physician training. Patients expect to receive safe, high-quality care in the nation’s teaching hospitals. Because it is their safety that is at issue, their voices should be central in policy decisions affecting patient safety. It is likewise important to integrate the perspectives of resident physicians, policy makers, and other constituencies in designing new policies. However, since its release, discussion of the Institute of Medicine report has been largely confined to the medical education community, led by the Accreditation Council for Graduate Medical Education (ACGME).
To begin gathering these perspectives and developing a plan to implement safer work hours for resident physicians, a conference entitled “Enhancing sleep, supervision and safety: What will it take to implement the Institute of Medicine recommendations?” was held at Harvard Medical School on June 17–18, 2010. This White Paper is a product of a diverse group of 26 representative stakeholders bringing relevant new information and innovative practices to bear on a critical patient safety problem. Given that our conference included experts from across disciplines with diverse perspectives and interests, not every recommendation was endorsed by each invited conference participant. However, every recommendation made here was endorsed by the majority of the group, and many were endorsed unanimously. Conference members participated in the process, reviewed the final product, and provided input before publication. Participants provided their individual perspectives, which do not necessarily represent the formal views of any organization.
In September 2010 the ACGME issued new rules to go into effect on July 1, 2011. Unfortunately, they stop considerably short of the Institute of Medicine’s recommendations and those endorsed by this conference. In particular, the ACGME only applied the limitation of 16 hours to first-year resident physicans. Thus, it is clear that policymakers, hospital administrators, and residency program directors who wish to implement safer health care systems must go far beyond what the ACGME will require. We hope this White Paper will serve as a guide and provide encouragement for that effort.
Resident physician workload and supervision
By the end of training, a resident physician should be able to practice independently. Yet much of resident physicians’ time is dominated by tasks with little educational value. The caseload can be so great that inadequate reflective time is left for learning based on clinical experiences. In addition, supervision is often vaguely defined and discontinuous. Medical malpractice data indicate that resident physicians are frequently named in lawsuits, most often for lack of supervision. The recommendations are: The ACGME should adjust resident physicians workload requirements to optimize educational value. Resident physicians as well as faculty should be involved in work redesign that eliminates nonessential and noneducational activity from resident physician dutiesMechanisms should be developed for identifying in real time when a resident physician’s workload is excessive, and processes developed to activate additional providersTeamwork should be actively encouraged in delivery of patient care. Historically, much of medical training has focused on individual knowledge, skills, and responsibility. As health care delivery has become more complex, it will be essential to train resident and attending physicians in effective teamwork that emphasizes collective responsibility for patient care and recognizes the signs, both individual and systemic, of a schedule and working conditions that are too demanding to be safeHospitals should embrace the opportunities that resident physician training redesign offers. Hospitals should recognize and act on the potential benefits of work redesign, eg, increased efficiency, reduced costs, improved quality of care, and resident physician and attending job satisfactionAttending physicians should supervise all hospital admissions. Resident physicians should directly discuss all admissions with attending physicians. Attending physicians should be both cognizant of and have input into the care patients are to receive upon admission to the hospitalInhouse supervision should be required for all critical care services, including emergency rooms, intensive care units, and trauma services. Resident physicians should not be left unsupervised to care for critically ill patients. In settings in which the acuity is high, physicians who have completed residency should provide direct supervision for resident physicians. Supervising physicians should always be physically in the hospital for supervision of resident physicians who care for critically ill patientsThe ACGME should explicitly define “good” supervision by specialty and by year of training. Explicit requirements for intensity and level of training for supervision of specific clinical scenarios should be providedCenters for Medicare and Medicaid Services (CMS) should use graduate medical education funding to provide incentives to programs with proven, effective levels of supervision. Although this action would require federal legislation, reimbursement rules would help to ensure that hospitals pay attention to the importance of good supervision and require it from their training programs
Resident physician work hours
Although the IOM “Sleep, supervision and safety” report provides a comprehensive review and discussion of all aspects of graduate medical education training, the report’s focal point is its recommendations regarding the hours that resident physicians are currently required to work. A considerable body of scientific evidence, much of it cited by the Institute of Medicine report, describes deteriorating performance in fatigued humans, as well as specific studies on resident physician fatigue and preventable medical errors.
The question before this conference was what work redesign and cultural changes are needed to reform work hours as recommended by the Institute of Medicine’s evidence-based report? Extensive scientific data demonstrate that shifts exceeding 12–16 hours without sleep are unsafe. Several principles should be followed in efforts to reduce consecutive hours below this level and achieve safer work schedules. The recommendations are: Limit resident physician work hours to 12–16 hour maximum shiftsA minimum of 10 hours off duty should be scheduled between shiftsResident physician input into work redesign should be actively solicitedSchedules should be designed that adhere to principles of sleep and circadian science; this includes careful consideration of the effects of multiple consecutive night shifts, and provision of adequate time off after night work, as specified in the IOM reportResident physicians should not be scheduled up to the maximum permissible limits; emergencies frequently occur that require resident physicians to stay longer than their scheduled shifts, and this should be anticipated in scheduling resident physicians’ work shiftsHospitals should anticipate the need for iterative improvement as new schedules are initiated; be prepared to learn from the initial phase-in, and change the plan as neededAs resident physician work hours are redesigned, attending physicians should also be considered; a potential consequence of resident physician work hour reduction and increased supervisory requirements may be an increase in work for attending physicians; this should be carefully monitored, and adjustments to attending physician work schedules made as needed to prevent unsafe work hours or working conditions for this group“Home call” should be brought under the overall limits of working hours; work load and hours should be monitored in each residency program to ensure that resident physicians and fellows on home call are getting sufficient sleepMedicare funding for graduate medical education in each hospital should be linked with adherence to the Institute of Medicine limits on resident physician work hours
Moonlighting by resident physicians
The Institute of Medicine report recommended including external as well as internal moonlighting in working hour limits. The recommendation is: All moonlighting work hours should be included in the ACGME working hour limits and actively monitored. Hospitals should formalize a moonlighting policy and establish systems for actively monitoring resident physician moonlighting
Safety of resident physicians
The “Sleep, supervision and safety” report also addresses fatigue-related harm done to resident physicians themselves. The report focuses on two main sources of physical injury to resident physicians impaired by fatigue, ie, needle-stick exposure to blood-borne pathogens and motor vehicle crashes. Providing safe transportation home for resident physicians is a logistical and financial challenge for hospitals. Educating physicians at all levels on the dangers of fatigue is clearly required to change driving behavior so that safe hospital-funded transport home is used effectively. Fatigue-related injury prevention (including not driving while drowsy) should be taught in medical school and during residency, and reinforced with attending physicians; hospitals and residency programs must be informed that resident physicians’ ability to judge their own level of impairment is impaired when they are sleep deprived; hence, leaving decisions about the capacity to drive to impaired resident physicians is not recommendedHospitals should provide transportation to all resident physicians who report feeling too tired to drive safely; in addition, although consecutive work should not exceed 16 hours, hospitals should provide transportation for all resident physicians who, because of unforeseen reasons or emergencies, work for longer than consecutive 24 hours; transportation under these circumstances should be automatically provided to house staff, and should not rely on self-identification or request
Training in effective handovers and quality improvement
Handover practice for resident physicians, attendings, and other health care providers has long been identified as a weak link in patient safety throughout health care settings. Policies to improve handovers of care must be tailored to fit the appropriate clinical scenario, recognizing that information overload can also be a problem. At the heart of improving handovers is the organizational effort to improve quality, an effort in which resident physicians have typically been insufficiently engaged. The recommendations are: Hospitals should train attending and resident physicians in effective handovers of careHospitals should create uniform processes for handovers that are tailored to meet each clinical setting; all handovers should be done verbally and face-to-face, but should also utilize written toolsWhen possible, hospitals should integrate hand-over tools into their electronic medical records (EMR) systems; these systems should be standardized to the extent possible across residency programs in a hospital, but may be tailored to the needs of specific programs and services; federal government should help subsidize adoption of electronic medical records by hospitals to improve signoutWhen feasible, handovers should be a team effort including nurses, patients, and familiesHospitals should include residents in their quality improvement and patient safety efforts; the ACGME should specify in their core competency requirements that resident physicians work on quality improvement projects; likewise, the Joint Commission should require that resident physicians be included in quality improvement and patient safety programs at teaching hospitals; hospital administrators and residency program directors should create opportunities for resident physicians to become involved in ongoing quality improvement projects and root cause analysis teams; feedback on successful quality improvement interventions should be shared with resident physicians and broadly disseminatedQuality improvement/patient safety concepts should be integral to the medical school curriculum; medical school deans should elevate the topics of patient safety, quality improvement, and teamwork; these concepts should be integrated throughout the medical school curriculum and reinforced throughout residency; mastery of these concepts by medical students should be tested on the United States Medical Licensing Examination (USMLE) stepsFederal government should support involvement of resident physicians in quality improvement efforts; initiatives to improve quality by including resident physicians in quality improvement projects should be financially supported by the Department of Health and Human Services
Monitoring and oversight of the ACGME
While the ACGME is a key stakeholder in residency training, external voices are essential to ensure that public interests are heard in the development and monitoring of standards. Consequently, the Institute of Medicine report recommended external oversight and monitoring through the Joint Commission and Centers for Medicare and Medicaid Services (CMS). The recommendations are: Make comprehensive fatigue management a Joint Commission National Patient Safety Goal; fatigue is a safety concern not only for resident physicians, but also for nurses, attending physicians, and other health care workers; the Joint Commission should seek to ensure that all health care workers, not just resident physicians, are working as safely as possibleFederal government, including the Centers for Medicare and Medicaid Services and the Agency for Healthcare Research and Quality, should encourage development of comprehensive fatigue management programs which all health systems would eventually be required to implementMake ACGME compliance with working hours a “ condition of participation” for reimbursement of direct and indirect graduate medical education costs; financial incentives will greatly increase the adoption of and compliance with ACGME standards
Future financial support for implementation
The Institute of Medicine’s report estimates that $1.7 billion (in 2008 dollars) would be needed to implement its recommendations. Twenty-five percent of that amount ($376 million) will be required just to bring hospitals into compliance with the existing 2003 ACGME rules. Downstream savings to the health care system could potentially result from safer care, but these benefits typically do not accrue to hospitals and residency programs, who have been asked historically to bear the burden of residency reform costs. The recommendations are: The Institute of Medicine should convene a panel of stakeholders, including private and public funders of health care and graduate medical education, to lay down the concrete steps necessary to identify and allocate the resources needed to implement the recommendations contained in the IOM “Resident duty hours: Enhancing sleep, supervision and safety” report. Conference participants suggested several approaches to engage public and private support for this initiativeEfforts to find additional funding to implement the Institute of Medicine recommendations should focus more broadly on patient safety and health care delivery reform; policy efforts focused narrowly upon resident physician work hours are less likely to succeed than broad patient safety initiatives that include residency redesign as a key componentHospitals should view the Institute of Medicine recommendations as an opportunity to begin resident physician work redesign projects as the core of a business model that embraces safety and ultimately saves resourcesBoth the Secretary of Health and Human Services and the Director of the Centers for Medicare and Medicaid Services should take the Institute of Medicine recommendations into consideration when promulgating rules for innovation grantsThe National Health Care Workforce Commission should consider the Institute of Medicine recommendations when analyzing the nation’s physician workforce needs
Recommendations for future research
Conference participants concurred that convening the stakeholders and agreeing on a research agenda was key. Some observed that some sectors within the medical education community have been reluctant to act on the data. Several logical funders for future research were identified. But above all agencies, Centers for Medicare and Medicaid Services is the only stakeholder that funds graduate medical education upstream and will reap savings downstream if preventable medical errors are reduced as a result of reform of resident physician work hours.
doi:10.2147/NSS.S19649
PMCID: PMC3630963  PMID: 23616719
resident; hospital; working hours; safety
18.  Exploring Situational Awareness in Diagnostic Errors in Primary Care 
BMJ quality & safety  2011;21(1):30-38.
Objective
Diagnostic errors in primary care are harmful but poorly studied. To facilitate understanding of diagnostic errors in real-world primary care settings using electronic health records (EHRs), this study explored the use of the Situational Awareness (SA) framework from aviation human factors research.
Methods
A mixed-methods study was conducted involving reviews of EHR data followed by semi-structured interviews of selected providers from two institutions in the US. The study population included 380 consecutive patients with colorectal and lung cancers diagnosed between February 2008 and January 2009. Using a pre-tested data collection instrument, trained physicians identified diagnostic errors, defined as lack of timely action on one or more established indications for diagnostic work-up for lung and colorectal cancers. Twenty-six providers involved in cases with and without errors were interviewed. Interviews probed for providers' lack of SA and how this may have influenced the diagnostic process.
Results
Of 254 cases meeting inclusion criteria, errors were found in 30 (32.6%) of 92 lung cancer cases and 56 (33.5%) of 167 colorectal cancer cases. Analysis of interviews related to error cases revealed evidence of lack of one of four levels of SA applicable to primary care practice: information perception, information comprehension, forecasting future events, and choosing appropriate action based on the first three levels. In cases without error, the application of the SA framework provided insight into processes involved in attention management.
Conclusions
A framework of SA can help analyze and understand diagnostic errors in primary care settings that use EHRs.
doi:10.1136/bmjqs-2011-000310
PMCID: PMC3692739  PMID: 21890757
diagnostic error; decision-making; patient safety; primary care; medical errors; human factors; cancer; electronic health records; diagnostic delays
19.  Telehealth research 
Aims
Oxleas Telehealth is piloting the effectiveness of remotely monitoring Tier 3 COPD1 patients to reduce the hospital admissions and enable community nurses to reach more patients. Setting alert thresholds for clinical readings, which indicate an ill patient if exceeded, is crucial to remote monitoring. Hitherto, thresholds have been set from national guidelines. Patient specific adjustments are then made upon periodic qualitative assessments of progress. This causes the following problems:
Excessive red alerts taking long time to clinically triage
No standardised method for adjusting thresholds
Anticipated operational issues when upscaling
Oxleas is researching a quantitative method for setting thresholds by statistically analysing daily variation in patients’ clinical measurements, to differentiate between common place variation in physiology, and special case variation due to ill-health.
Potential benefits
Reduced nurse time spent triaging data and contacting patients with red alerts but well
Freed up nursing time to deliver more community care to patients
Efficient method for threshold setting when upscaling
Cycle 1 method
For each patient, pulse, oxygen and blood pressure readings were collated for the two preceding months. Arithmetic means were calculated for each. Spread analysis confirmed a bell shaped distribution. Alert thresholds were set three standard deviations from mean for each parameter. Exceptions were made for variation indicating good health (e.g. high oxygen levels). Episodes of abnormal readings were compared to clinical notes to find correlation with episodes of illness.
Cycle 1 result
Correlation revealed 100% sensitivity, where all episodes of illness produced measurements exceeding thresholds. Specificity was poor (85% for oxygen, 60% for pulse and 52% for blood pressure) with numerous false positive readings. Of false positives, 61% occurred due to technical error (mostly faulty batteries or poor user technique). False positives for non-technical reasons were 78% for blood pressure, 57% for pulse and 20% for oxygen.
Cycle 2 method
This was performed to audit technical error. From cycle 1 results, eliminating technical error would improve specificity to 86% for oxygen, 73% for pulse and 63% for blood pressure. Drop down menus were installed on the triage software to code for:
Number of abnormal measurements daily
Number of technical errors daily (separate codes for battery error and patient user errors)
These were selected daily for each patient. Concurrently, improved systems for battery replacement have been implemented. Patient training has been provided when indicated. Alert thresholds have been updated as in cycle one.
Cycle 2 result (due December)
Early signs of reduced triage time and fewer red alerts will be quantified.
Conclusions
Statistical analysis of measurement variation can provide a highly sensitive method for setting alert thresholds that accurately identifying illness however technical need reduced to improve specificity. The method is most effective for oxygen saturation. Improving effectiveness for other parameters may require improved patient training for equipment use (e.g. activities immediately prior to taking readings etc.). Research cycles using narrower standard deviations to improve specificity are required. If high sensitivity and specificity can be achieved, method can be used to automatically adjust thresholds periodically, reflecting the patient’s most recent clinical status.
PMCID: PMC3571163
telehealth; COPD
20.  A preliminary taxonomy of medical errors in family practice 
Quality & safety in health care  2002;11(3):233-238.
Objective: To develop a preliminary taxonomy of primary care medical errors.
Design: Qualitative analysis to identify categories of error reported during a randomized controlled trial of computer and paper reporting methods.
Setting: The National Network for Family Practice and Primary Care Research.
Participants: Family physicians.
Main outcome measures: Medical error category, context, and consequence.
Results: Forty two physicians made 344 reports: 284 (82.6%) arose from healthcare systems dysfunction; 46 (13.4%) were errors due to gaps in knowledge or skills; and 14 (4.1%) were reports of adverse events, not errors. The main subcategories were: administrative failures (102; 30.9% of errors), investigation failures (82; 24.8%), treatment delivery lapses (76; 23.0%), miscommunication (19; 5.8%), payment systems problems (4; 1.2%), error in the execution of a clinical task (19; 5.8%), wrong treatment decision (14; 4.2%), and wrong diagnosis (13; 3.9%). Most reports were of errors that were recognized and occurred in reporters' practices. Affected patients ranged in age from 8 months to 100 years, were of both sexes, and represented all major US ethnic groups. Almost half the reports were of events which had adverse consequences. Ten errors resulted in patients being admitted to hospital and one patient died.
Conclusions: This medical error taxonomy, developed from self-reports of errors observed by family physicians during their routine clinical practice, emphasizes problems in healthcare processes and acknowledges medical errors arising from shortfalls in clinical knowledge and skills. Patient safety strategies with most effect in primary care settings need to be broader than the current focus on medication errors.
doi:10.1136/qhc.11.3.233
PMCID: PMC1743626  PMID: 12486987
21.  Intrastromal Corneal Ring Implants for Corneal Thinning Disorders 
Executive Summary
Objective
The purpose of this project was to determine the role of corneal implants in the management of corneal thinning disease conditions. An evidence-based review was conducted to determine the safety, effectiveness and durability of corneal implants for the management of corneal thinning disorders. The evolving directions of research in this area were also reviewed.
Subject of the Evidence-Based Analysis
The primary treatment objectives for corneal implants are to normalize corneal surface topography, improve contact lens tolerability, and restore visual acuity in order to delay or defer the need for corneal transplant. Implant placement is a minimally invasive procedure that is purported to be safe and effective. The procedure is also claimed to be adjustable, reversible, and both eyes can be treated at the same time. Further, implants do not limit the performance of subsequent surgical approaches or interfere with corneal transplant. The evidence for these claims is the focus of this review.
The specific research questions for the evidence review were as follows:
Safety
Corneal Surface Topographic Effects:
Effects on corneal surface remodelling
Impact of these changes on subsequent interventions, particularly corneal transplantation (penetrating keratoplasty [PKP])
Visual Acuity
Refractive Outcomes
Visual Quality (Symptoms): such as contrast vision or decreased visual symptoms (halos, fluctuating vision)
Contact lens tolerance
Functional visual rehabilitation and quality of life
Patient satisfaction:
Disease Process:
Impact on corneal thinning process
Effect on delaying or deferring the need for corneal transplantation
Clinical Need: Target Population and Condition
Corneal ectasia (thinning) comprises a range of disorders involving either primary disease conditions such as keratoconus and pellucid marginal corneal degeneration or secondary iatrogenic conditions such as corneal thinning occurring after LASIK refractive surgery. The condition occurs when the normally round dome-shaped cornea progressively thins causing a cone-like bulge or forward protrusion in response to the normal pressure of the eye. Thinning occurs primarily in the stoma layers and is believed to be a breakdown in the collagen network. This bulging can lead to an irregular shape or astigmatism of the cornea and, because the anterior part of the cornea is largely responsible for the focusing of light on the retina, results in loss of visual acuity. This can make even simple daily tasks, such as driving, watching television or reading, difficult to perform.
Keratoconus (KC) is the most common form of corneal thinning disorder and is a noninflammatory chronic disease process. Although the specific causes of the biomechanical alterations that occur in KC are unknown, there is a growing body of evidence to suggest that genetic factors may play an important role. KC is a rare condition (<0.05% of the population) and is unique among chronic eye diseases as it has an early age of onset (median age of 25 years). Disease management for this condition follows a step-wise approach depending on disease severity. Contact lenses are the primary treatment of choice when there is irregular astigmatism associated with the disease. When patients can no longer tolerate contact lenses or when lenses no longer provide adequate vision, patients are referred for corneal transplant.
Keratoconus is one of the leading indications for corneal transplants and has been so for the last three decades. Yet, despite high graft survival rates of up to 20 years, there are reasons to defer receiving transplants for as long as possible. Patients with keratoconus are generally young and life-long term graft survival would be an important consideration. The surgery itself involves lengthy time off work and there are potential complications from long term steroid use following surgery, as well as the risk of developing secondary cataracts, glaucoma etc. After transplant, recurrent KC is possible with need for subsequent intervention. Residual refractive errors and astigmatism can remain challenging after transplantation and high refractive surgery rates and re-graft rates in KC patients have been reported. Visual rehabilitation or recovery of visual acuity after transplant may be slow and/or unsatisfactory to patients.
Description of Technology/Therapy
INTACS® (Addition Technology Inc. Sunnyvale, CA, formerly KeraVision, Inc.) are the only currently licensed corneal implants in Canada. The implants are micro-thin poly methyl methacrylate crescent shaped ring segments with a circumference arc length of 150 degrees, an external diameter of 8.10 mm, an inner diameter of 6.77 mm, and a range of different thicknesses. Implants act as passive spacers and, when placed in the cornea, cause local separation of the corneal lamellae resulting in a shortening of the arc length of the anterior corneal curvature and flattening the central cornea. Increasing segment thickness results in greater lamellar separation with increased flattening of the cornea correcting for myopia by decreasing the optical power of the eye. Corneal implants also improve corneal astigmatism but the mechanism of action for this is less well understood.
Treatment with corneal implants is considered for patients who are contact lens intolerant, having adequate corneal thickness particularly around the area of the implant incision site and without central corneal scarring. Those with central corneal scarring would not benefit from implants and those without an adequate corneal thickness, particularly in the region that the implants are being inserted, would be at increased risk for corneal perforation. Patients desiring to have visual rehabilitation that does not include glasses or contact lenses would not be candidates for corneal ring implants.
Placement of the implants is an outpatient procedure with topical anesthesia generally performed by either corneal specialists or refractive surgeons. It involves creating tunnels in the corneal stroma to secure the implants either by a diamond knife or laser calibrated to an approximate depth of 70% of the cornea. Variable approaches have been employed by surgeons in selecting ring segment size, number and position. Generally, two segments of equal thickness are placed superiorly and inferiorly to manage symmetrical patterns of corneal thinning whereas one segment may be placed to manage asymmetric thinning patterns.
Following implantation, the major safety concerns are for potential adverse events including corneal perforation, infection, corneal infiltrates, corneal neovascularization, ring migration and extrusion and corneal thinning. Technical results can be unsatisfactory for several reasons. Treatment may result in an over or under-correction of refraction and may induce astigmatism or asymmetry of the cornea.
Progression of the corneal cone with corneal opacities is also invariably an indication for progression to corneal transplant. Other reasons for treatment failure or patient dissatisfaction include foreign body sensation, unsatisfactory visual quality with symptoms such as double vision, fluctuating vision, poor night vision or visual side effects related to ring edge or induced or unresolved astigmatism.
Evidence-Based Analysis Methods
The literature search strategy employed keywords and subject headings to capture the concepts of 1) intrastromal corneal rings and 2) corneal diseases, with a focus on keratoconus, astigmatism, and corneal ectasia. The initial search was run on April 17, 2008, and a final search was run on March 6, 2009 in the following databases: Ovid MEDLINE (1996 to February Week 4 2009), OVID MEDLINE In-Process and Other Non-Indexed Citations, EMBASE (1980 to 2009 Week 10), OVID Cochrane Library, and the Centre for Reviews and Dissemination/International Agency for Health Technology Assessment. Parallel search strategies were developed for the remaining databases. Search results were limited to human and English-language published between January 2000 and April 17, 2008. The resulting citations were downloaded into Reference Manager, v.11 (ISI Researchsoft, Thomson Scientific, U.S.A), and duplicates were removed. The Web sites of several other health technology agencies were also reviewed including the Canadian Agency for Drugs and Technologies in Health (CADTH), ECRI, and the United Kingdom National Institute for Clinical Excellence (NICE). The bibliographies of relevant articles were scanned.
Inclusion Criteria
English language reports and human studies
Any corneal thinning disorder
Reports with corneal implants used alone or in conjunction with other interventions
Original reports with defined study methodology
Reports including standardized measurements on outcome events such as technical success, safety, effectiveness, durability, vision quality of life or patient satisfaction
Case reports or case series for complications and adverse events
Exclusion Criteria
Non-systematic reviews, letters, comments and editorials
Reports not involving outcome events such as safety, effectiveness, durability, vision quality or patient satisfaction following an intervention with corneal implants
Reports not involving corneal thinning disorders and an intervention with corneal implants
Summary of Findings
In the MAS evidence review on intrastromal corneal ring implants, 66 reports were identified on the use of implants for management of corneal thinning disorders. Reports varied according to their primary clinical indication, type of corneal implant, and whether or not secondary procedures were used in conjunction with the implants. Implants were reported to manage post LASIK thinning and/or uncorrected refractive error and were also reported as an adjunctive intervention both during and after corneal transplant to manage recurrent thinning and/or uncorrected refractive error.
Ten pre-post cohort longitudinal follow-up studies were identified examining the safety and effectiveness of INTAC corneal implants in patients with keratoconus. Five additional cohort studies were identified using the Ferrara implant for keratoconus management but because this corneal implant is not licensed in Canada these studies were not reviewed.
The cohorts implanted with INTACS involved 608 keratoconus patients (754 eyes) followed for 1, 2 or 3 years. Three of the reports involved ≥ 2 years of follow-up with the longest having 5-year follow-up data for a small number of patients. Four of the INTAC cohort studies involved 50 or more patients; the largest involved 255 patients. Inclusion criteria for the studies were consistent and included patients who were contact lens intolerant, had adequate corneal thickness, particularly around the area of the implant incision site, and without central corneal scarring. Disease severity, thinning pattern, and corneal cone protrusions all varied and generally required different treatment approaches involving defined segment sizes and locations.
A wide range of outcome measures were reported in the cohort studies. High levels of technical success or ability to place INTAC segments were reported. Technically related complications were often delayed and generally reported as segment migration attributable to early experience. Overall, complications were infrequently reported and largely involved minor reversible events without clinical sequelae.
The outcomes reported across studies involved statistically significant and clinically relevant improvements in corneal topography, refraction and visual acuity, for both uncorrected and best-corrected visual acuity. Patients’ vision was usually restored to within normal functioning levels and for those not achieving satisfactory correction, insertion of intraocular lenses was reported in case studies to result in additional gains in visual acuity. Vision loss (infrequently reported) was usually reversed by implant exchange or removal. The primary effects of INTACS on corneal surface remodelling were consistent with secondary improvements in refractive error and visual acuity. The improvements in visual acuity and refractive error noted at 6 months were maintained at 1 and 2-year follow-up
Improvements in visual acuity and refractive error following insertion of INTACS, however, were not noted for all patients. Although improvements were not found to vary across age groups there were differences across stages of disease. Several reports suggested that improvements in visual acuity and refractive outcomes may not be as large or predictable in more advanced stages of KC. Some studies have suggested that the effects of INTACs were much greater in flattening the corneal surface than in correcting astigmatism. However, these studies involved small numbers of high risk patients in advanced stages of KC and conclusions made from this group are limited.
INTACS were used for other indications other than primary KC. The results of implant insertion on corneal topography, refraction, and visual acuity in post-LASIK thinning cases were similar to those reported for KC. The evidence for this indication, however, only involved case reports and small case series. INTACS were also successfully used to treat recurrent KC after corneal transplant but this was based on only a single case report. Corneal implants were compared to corneal transplantation but these studies were not randomized and based on small numbers of selected patients.
The foremost limitation of the evidence base is the basic study design in the reports that involved longitudinal follow-up only for the treated group; there were no randomized trials. Follow-up in the trials (although at prescribed intervals) often had incomplete accounts of losses at follow-up and estimates of change were often not reported or based on group differences. Second, although standardized outcome measures were reported, contact lens tolerance (a key treatment objective) was infrequently specified. A third general limitation was the lack of reporting of patients’ satisfaction with their vision quality or functional vision. Outcome measures for vision quality and impact on patient quality of life were available but rarely reported and have been noted to be a limitation in ophthalmological literature in general. Fourth, the longitudinal cohort studies have not followed patients long enough to evaluate the impact of implants on the underlying disease process (follow-up beyond 3 years is limited). Additionally, only a few of these studies directly examined corneal thinning in follow-up. The overall quality of evidence determined using the GRADE hierarchy of evidence was moderate.
There is some evidence in these studies to support the claim that corneal implants do not interfere with, or increase the difficultly of, subsequent corneal transplant, at least for those performed shortly after INTAC placement. Although it’s uncertain for how long implants can delay the need for a corneal transplant, given that patients with KC are often young (in their twenties and thirties), delaying transplant for any number of years may still be a valuable consideration.
Conclusion
The clinical indications for corneal implants have evolved from management of myopia in normal eyes to the management of corneal thinning disorders such as KC and thinning occurring after refractive surgery. Despite the limited evidence base for corneal implants, which consists solely of longitudinal follow-up studies, they appear to be a valuable clinical tool for improving vision in patients with corneal thinning. For patients unable to achieve functional vision, corneal implants achieved statistically significant and clinically relevant improvements in corneal topography, refraction, and visual acuity, providing a useful alternative to corneal transplant. Implants may also have a rescue function, treating corneal thinning occurring after refractive surgery in normal eyes, or managing refractive errors following corneal transplant. The treatment offers several advantages in that it’s an outpatient based procedure, is associated with minimal risk, and has high technical success rates. Both eyes can be treated at once and the treatment is adjustable and reversible. The implants can be removed or exchanged to improve vision without limiting subsequent interventions, particularly corneal transplant.
Better reporting on vision quality, functional vision and patient satisfaction, however, would improve evaluation of the impact of these devices. Information on the durability of the implants’ treatment effects and their affects on underlying disease processes is limited. This information is becoming more important as alternative treatment strategies, such as collagen cross-linking aimed at strengthening the underlying corneal tissue, are emerging and which might prove to be more effective or increase the effectiveness of the implants, particularly in advances stages of corneal thinning.
Ontario Health System Considerations
At present there are approximately 70 ophthalmologists in Canada who’ve had training with corneal implants; 30 of these practice in Ontario. Industry currently sponsors the training, proctoring and support for the procedure. The cost of the implant device ranges from $950 to $1200 (CAD) and costs for instrumentation range from $20,000 to $30,000 (CAD) (a one time capital expenditure). There is no physician services fee code for corneal implants in Ontario but assuming that they are no higher than those for a corneal transplant, the estimated surgical costs would be $914.32(CAD) An estimated average cost per patient, based on device costs and surgical fees, for treatment is $1,964 (CAD) (range $1,814 to $2,114) per eye. There have also been no out of province treatment requests. In Ontario the treatment is currently being offered in private clinics and an increasing number of ophthalmologists are being certified in the technique by the manufacturer.
KC is a rare disease and not all of these patients would be eligible candidates for treatment with corneal implants. Based on published population rates of KC occurrence, it can be expected that there is a prevalent population of approximately 6,545 patients and an incident population of 240 newly diagnosed cases per year. Given this small number of potential cases, the use of corneal implants would not be expected to have much impact on the Ontario healthcare system. The potential impact on the provincial budget for managing the incident population, assuming the most conservative scenario (i.e., all are eligible and all receive bilateral implants) ranges from $923 thousand to $1.1 million (CAD). This estimate would vary based on a variety of criteria including eligibility, unilateral or bilateral interventions, re-interventions, capacity and uptake
Keywords
Keratoconus, corneal implants, corneal topography, corneal transplant, visual acuity, refractive error
PMCID: PMC3385416  PMID: 23074513
22.  Impact of e-alert for detection of acute kidney injury on processes of care and outcomes: protocol for a systematic review and meta-analysis 
BMJ Open  2016;6(5):e011152.
Introduction
Acute kidney injury (AKI) is a common complication in hospitalised patients. It imposes significant risk for major morbidity and mortality. Moreover, patients suffering an episode of AKI consume considerable health resources. Recently, a number of studies have evaluated the implementation of automated electronic alerts (e-alerts) configured from electronic medical records (EMR) and clinical information systems (CIS) to warn healthcare providers of early or impending AKI in hospitalised patients. The impact of e-alerts on care processes, patient outcomes and health resource use, however, remains uncertain.
Methods and analysis
We will perform a systematic review to describe and appraise e-alerts for AKI, and evaluate their impact on processes of care, clinical outcomes and health services use. In consultation with a research librarian, a search strategy will be developed and electronic databases (ie, MEDLINE, EMBASE, CINAHL, Cochrane Library and Inspec via Engineering Village) searched. Selected grey literature sources will also be searched. Search themes will focus on e-alerts and AKI. Citation screening, selection, quality assessment and data abstraction will be performed in duplicate. The primary analysis will be narrative; however, where feasible, pooled analysis will be performed. Each e-alert will be described according to trigger, type of alert, target recipient and degree of intrusiveness. Pooled effect estimates will be described, where applicable.
Ethics and dissemination
Our systematic review will synthesise the literature on the value of e-alerts to detect AKI, and their impact on processes, patient-centred outcomes and resource use, and also identify key knowledge gaps and barriers to implementation. This is a fundamental step in a broader research programme aimed to understand the ideal structure of e-alerts, target population and methods for implementation, to derive benefit. Research ethics approval is not required for this review.
Systematic review registration number
CRD42016033033.
doi:10.1136/bmjopen-2016-011152
PMCID: PMC4861089  PMID: 27150187
electronic alerts; computerized decision support; acute kidney injury
23.  Fatal Dengue Hemorrhagic Fever in Adults: Emphasizing the Evolutionary Pre-fatal Clinical and Laboratory Manifestations 
Background
A better description of the clinical and laboratory manifestations of fatal patients with dengue hemorrhagic fever (DHF) is important in alerting clinicians of severe dengue and improving management.
Methods and Findings
Of 309 adults with DHF, 10 fatal patients and 299 survivors (controls) were retrospectively analyzed. Regarding causes of fatality, massive gastrointestinal (GI) bleeding was found in 4 patients, dengue shock syndrome (DSS) alone in 2; DSS/subarachnoid hemorrhage, Klebsiella pneumoniae meningitis/bacteremia, ventilator associated pneumonia, and massive GI bleeding/Enterococcus faecalis bacteremia each in one. Fatal patients were found to have significantly higher frequencies of early altered consciousness (≤24 h after hospitalization), hypothermia, GI bleeding/massive GI bleeding, DSS, concurrent bacteremia with/without shock, pulmonary edema, renal/hepatic failure, and subarachnoid hemorrhage. Among those experienced early altered consciousness, massive GI bleeding alone/with uremia/with E. faecalis bacteremia, and K. pneumoniae meningitis/bacteremia were each found in one patient. Significantly higher proportion of bandemia from initial (arrival) laboratory data in fatal patients as compared to controls, and higher proportion of pre-fatal leukocytosis and lower pre-fatal platelet count as compared to initial laboratory data of fatal patients were found. Massive GI bleeding (33.3%) and bacteremia (25%) were the major causes of pre-fatal leukocytosis in the deceased patients; 33.3% of the patients with pre-fatal profound thrombocytopenia (<20000/µL), and 50% of the patients with pre-fatal prothrombin time (PT) prolongation experienced massive GI bleeding.
Conclusions
Our report highlights causes of fatality other than DSS in patients with severe dengue, and suggested hypothermia, leukocytosis and bandemia may be warning signs of severe dengue. Clinicians should be alert to the potential development of massive GI bleeding, particularly in patients with early altered consciousness, profound thrombocytopenia, prolonged PT and/or leukocytosis. Antibiotic(s) should be empirically used for patients at risk for bacteremia until it is proven otherwise, especially in those with early altered consciousness and leukocytosis.
Author Summary
Fatality rate and causes of fatality in dengue-affected patients greatly varied from one reported series to another. A better understanding of the clinical and laboratory manifestations of fatal patients with dengue hemorrhagic fever (DHF) is important in alerting clinicians of severe dengue and improving management. In a retrospective analysis of 10 adults who died of and 299 survived (controls) DHF, dengue shock syndrome (DSS) alone was found in only 20% of dengue-related death, while intractable massive gastrointestinal (GI) bleeding was found in 40%, and DSS with concurrent subarachnoid hemorrhage, intractable massive GI bleeding with concurrent bacteremia, bacterial sepsis/meningitis, and sepsis due to ventilator associated pneumonia each were found in 10%. Early altered consciousness (developed ≤24 h after hospitalization), GI bleeding/massive GI bleeding and concurrent bacteremia were significantly found among the deceased patients. Our data suggest that hypothermia, leukocytosis and bandemia at hospital presentation may be warning signs of severe dengue. Clinicians should be alert to the potential development of massive GI bleeding, particularly in patients with early altered consciousness, profound thrombocytopenia, prothrombin time prolongation and/or leukocytosis. Antibiotic(s) should be empirically used for patients at risk for bacteremia until it is proven otherwise, especially in those with early altered consciousness and leukocytosis.
doi:10.1371/journal.pntd.0001532
PMCID: PMC3283557  PMID: 22363829
24.  Proceedings of the 3rd Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2015: advancing efficient methodologies through community partnerships and team science 
Lewis, Cara | Darnell, Doyanne | Kerns, Suzanne | Monroe-DeVita, Maria | Landes, Sara J. | Lyon, Aaron R. | Stanick, Cameo | Dorsey, Shannon | Locke, Jill | Marriott, Brigid | Puspitasari, Ajeng | Dorsey, Caitlin | Hendricks, Karin | Pierson, Andria | Fizur, Phil | Comtois, Katherine A. | Palinkas, Lawrence A. | Chamberlain, Patricia | Aarons, Gregory A. | Green, Amy E. | Ehrhart, Mark. G. | Trott, Elise M. | Willging, Cathleen E. | Fernandez, Maria E. | Woolf, Nicholas H. | Liang, Shuting Lily | Heredia, Natalia I. | Kegler, Michelle | Risendal, Betsy | Dwyer, Andrea | Young, Vicki | Campbell, Dayna | Carvalho, Michelle | Kellar-Guenther, Yvonne | Damschroder, Laura J. | Lowery, Julie C. | Ono, Sarah S. | Carlson, Kathleen F. | Cottrell, Erika K. | O’Neil, Maya E. | Lovejoy, Travis L. | Arch, Joanna J. | Mitchell, Jill L. | Lewis, Cara C. | Marriott, Brigid R. | Scott, Kelli | Coldiron, Jennifer Schurer | Bruns, Eric J. | Hook, Alyssa N. | Graham, Benjamin C. | Jordan, Katelin | Hanson, Rochelle F. | Moreland, Angela | Saunders, Benjamin E. | Resnick, Heidi S. | Stirman, Shannon Wiltsey | Gutner, Cassidy A. | Gamarra, Jennifer | Vogt, Dawne | Suvak, Michael | Wachen, Jennifer Schuster | Dondanville, Katherine | Yarvis, Jeffrey S. | Mintz, Jim | Peterson, Alan L. | Borah, Elisa V. | Litz, Brett T. | Molino, Alma | McCaughan, Stacey Young | Resick, Patricia A. | Pandhi, Nancy | Jacobson, Nora | Serrano, Neftali | Hernandez, Armando | Schreiter, Elizabeth Zeidler- | Wietfeldt, Natalie | Karp, Zaher | Pullmann, Michael D. | Lucenko, Barbara | Pavelle, Bridget | Uomoto, Jacqueline A. | Negrete, Andrea | Cevasco, Molly | Kerns, Suzanne E. U. | Franks, Robert P. | Bory, Christopher | Miech, Edward J. | Damush, Teresa M. | Satterfield, Jason | Satre, Derek | Wamsley, Maria | Yuan, Patrick | O’Sullivan, Patricia | Best, Helen | Velasquez, Susan | Barnett, Miya | Brookman-Frazee, Lauren | Regan, Jennifer | Stadnick, Nicole | Hamilton, Alison | Lau, Anna | Regan, Jennifer | Hamilton, Alison | Stadnick, Nicole | Barnett, Miya | Lau, Anna | Brookman-Frazee, Lauren | Stadnick, Nicole | Lau, Anna | Barnett, Miya | Regan, Jennifer | Roesch, Scott | Brookman-Frazee, Lauren | Powell, Byron J. | Waltz, Thomas J. | Chinman, Matthew J. | Damschroder, Laura | Smith, Jeffrey L. | Matthieu, Monica M. | Proctor, Enola K. | Kirchner, JoAnn E. | Waltz, Thomas J. | Powell, Byron J. | Chinman, Matthew J. | Damschroder, Laura J. | Smith, Jeffrey L. | Matthieu, Monica J. | Proctor, Enola K. | Kirchner, JoAnn E. | Matthieu, Monica M. | Rosen, Craig S. | Waltz, Thomas J. | Powell, Byron J. | Chinman, Matthew J. | Damschroder, Laura J. | Smith, Jeffrey L. | Proctor, Enola K. | Kirchner, JoAnn E. | Walker, Sarah C. | Bishop, Asia S. | Lockhart, Mariko | Rodriguez, Allison L. | Manfredi, Luisa | Nevedal, Andrea | Rosenthal, Joel | Blonigen, Daniel M. | Mauricio, Anne M. | Dishion, Thomas D. | Rudo-Stern, Jenna | Smith, Justin D. | Locke, Jill | Wolk, Courtney Benjamin | Harker, Colleen | Olsen, Anne | Shingledecker, Travis | Barg, Frances | Mandell, David | Beidas, Rinad S. | Hansen, Marissa C. | Aranda, Maria P. | Torres-Vigil, Isabel | Hartzler, Bryan | Steinfeld, Bradley | Gildred, Tory | Harlin, Zandrea | Shephard, Fredric | Ditty, Matthew S. | Doyle, Andrea | Bickel, John A. | Cristaudo, Katharine | Fox, Dan | Combs, Sonia | Lischner, David H. | Van Dorn, Richard A. | Tueller, Stephen J. | Hinde, Jesse M. | Karuntzos, Georgia T. | Monroe-DeVita, Maria | Peterson, Roselyn | Darnell, Doyanne | Berliner, Lucy | Dorsey, Shannon | Murray, Laura K. | Botanov, Yevgeny | Kikuta, Beverly | Chen, Tianying | Navarro-Haro, Marivi | DuBose, Anthony | Korslund, Kathryn E. | Linehan, Marsha M. | Harker, Colleen M. | Karp, Elizabeth A. | Edmunds, Sarah R. | Ibañez, Lisa V. | Stone, Wendy L. | Andrews, Jack H. | Johnides, Benjamin D. | Hausman, Estee M. | Hawley, Kristin M. | Prusaczyk, Beth | Ramsey, Alex | Baumann, Ana | Colditz, Graham | Proctor, Enola K. | Botanov, Yevgeny | Kikuta, Beverly | Chen, Tianying | Navarro-Haro, Marivi | DuBose, Anthony | Korslund, Kathryn E. | Linehan, Marsha M. | Harker, Colleen M. | Karp, Elizabeth A. | Edmunds, Sarah R. | Ibañez, Lisa V. | Stone, Wendy L. | Choy-Brown, Mimi | Andrews, Jack H. | Johnides, Benjamin D. | Hausman, Estee M. | Hawley, Kristin M. | Prusaczyk, Beth | Ramsey, Alex | Baumann, Ana | Colditz, Graham | Proctor, Enola K. | Meza, Rosemary D. | Dorsey, Shannon | Wiltsey-Stirman, Shannon | Sedlar, Georganna | Lucid, Leah | Dorsey, Caitlin | Marriott, Brigid | Zounlome, Nelson | Lewis, Cara | Gutner, Cassidy A. | Monson, Candice M. | Shields, Norman | Mastlej, Marta | Landy, Meredith SH | Lane, Jeanine | Stirman, Shannon Wiltsey | Finn, Natalie K. | Torres, Elisa M. | Ehrhart, Mark. G. | Aarons, Gregory A. | Malte, Carol A. | Lott, Aline | Saxon, Andrew J. | Boyd, Meredith | Scott, Kelli | Lewis, Cara C. | Pierce, Jennifer D. | Lorthios-Guilledroit, Agathe | Richard, Lucie | Filiatrault, Johanne | Hallgren, Kevin | Crotwell, Shirley | Muñoz, Rosa | Gius, Becky | Ladd, Benjamin | McCrady, Barbara | Epstein, Elizabeth | Clapp, John D. | Ruderman, Danielle E. | Barwick, Melanie | Barac, Raluca | Zlotkin, Stanley | Salim, Laila | Davidson, Marnie | Bunger, Alicia C. | Powell, Byron J. | Robertson, Hillary A. | Botsko, Christopher | Landes, Sara J. | Smith, Brandy N. | Rodriguez, Allison L. | Trent, Lindsay R. | Matthieu, Monica M. | Powell, Byron J. | Proctor, Enola K. | Harned, Melanie S. | Navarro-Haro, Marivi | Korslund, Kathryn E. | Chen, Tianying | DuBose, Anthony | Ivanoff, André | Linehan, Marsha M. | Garcia, Antonio R. | Kim, Minseop | Palinkas, Lawrence A. | Snowden, Lonnie | Landsverk, John | Sweetland, Annika C. | Fernandes, Maria Jose | Santos, Edilson | Duarte, Cristiane | Kritski, Afrânio | Krawczyk, Noa | Nelligan, Caitlin | Wainberg, Milton L. | Aarons, Gregory A. | Sommerfeld, David H. | Chi, Benjamin | Ezeanolue, Echezona | Sturke, Rachel | Kline, Lydia | Guay, Laura | Siberry, George | Bennett, Ian M. | Beidas, Rinad | Gold, Rachel | Mao, Johnny | Powers, Diane | Vredevoogd, Mindy | Unutzer, Jurgen | Schroeder, Jennifer | Volpe, Lane | Steffen, Julie | Dorsey, Shannon | Pullmann, Michael D | Kerns, Suzanne E. U. | Jungbluth, Nathaniel | Berliner, Lucy | Thompson, Kelly | Segell, Eliza | McGee-Vincent, Pearl | Liu, Nancy | Walser, Robyn | Runnals, Jennifer | Shaw, R. Keith | Landes, Sara J. | Rosen, Craig | Schmidt, Janet | Calhoun, Patrick | Varkovitzky, Ruth L. | Landes, Sara J. | Drahota, Amy | Martinez, Jonathan I. | Brikho, Brigitte | Meza, Rosemary | Stahmer, Aubyn C. | Aarons, Gregory A. | Williamson, Anna | Rubin, Ronnie M. | Powell, Byron J. | Hurford, Matthew O. | Weaver, Shawna L. | Beidas, Rinad S. | Mandell, David S. | Evans, Arthur C. | Powell, Byron J. | Beidas, Rinad S. | Rubin, Ronnie M. | Stewart, Rebecca E. | Wolk, Courtney Benjamin | Matlin, Samantha L. | Weaver, Shawna | Hurford, Matthew O. | Evans, Arthur C. | Hadley, Trevor R. | Mandell, David S. | Gerke, Donald R. | Prusaczyk, Beth | Baumann, Ana | Lewis, Ericka M. | Proctor, Enola K. | McWilliam, Jenna | Brown, Jacquie | Tucker, Michelle | Conte, Kathleen P | Lyon, Aaron R. | Boyd, Meredith | Melvin, Abigail | Lewis, Cara C. | Liu, Freda | Jungbluth, Nathaniel | Kotte, Amelia | Hill, Kaitlin A. | Mah, Albert C. | Korathu-Larson, Priya A. | Au, Janelle R. | Izmirian, Sonia | Keir, Scott | Nakamura, Brad J. | Higa-McMillan, Charmaine K. | Cooper, Brittany Rhoades | Funaiole, Angie | Dizon, Eleanor | Hawkins, Eric J. | Malte, Carol A. | Hagedorn, Hildi J. | Berger, Douglas | Frank, Anissa | Lott, Aline | Achtmeyer, Carol E. | Mariano, Anthony J. | Saxon, Andrew J. | Wolitzky-Taylor, Kate | Rawson, Richard | Ries, Richard | Roy-Byrne, Peter | Craske, Michelle | Simmons, Dena | Torrente, Catalina | Nathanson, Lori | Carroll, Grace | Smith, Justin D. | Brown, Kimbree | Ramos, Karina | Thornton, Nicole | Dishion, Thomas J. | Stormshak, Elizabeth A. | Shaw, Daniel S. | Wilson, Melvin N. | Choy-Brown, Mimi | Tiderington, Emmy | Smith, Bikki Tran | Padgett, Deborah K. | Rubin, Ronnie M. | Ray, Marilyn L. | Wandersman, Abraham | Lamont, Andrea | Hannah, Gordon | Alia, Kassandra A. | Hurford, Matthew O. | Evans, Arthur C. | Saldana, Lisa | Schaper, Holle | Campbell, Mark | Chamberlain, Patricia | Shapiro, Valerie B. | Kim, B.K. Elizabeth | Fleming, Jennifer L. | LeBuffe, Paul A. | Landes, Sara J. | Lewis, Cara C. | Rodriguez, Allison L. | Marriott, Brigid R. | Comtois, Katherine Anne | Lewis, Cara C. | Stanick, Cameo | Weiner, Bryan J. | Halko, Heather | Dorsey, Caitlin
Implementation Science : IS  2016;11(Suppl 1):85.
Table of contents
Introduction to the 3rd Biennial Conference of the Society for Implementation Research Collaboration: advancing efficient methodologies through team science and community partnerships
Cara Lewis, Doyanne Darnell, Suzanne Kerns, Maria Monroe-DeVita, Sara J. Landes, Aaron R. Lyon, Cameo Stanick, Shannon Dorsey, Jill Locke, Brigid Marriott, Ajeng Puspitasari, Caitlin Dorsey, Karin Hendricks, Andria Pierson, Phil Fizur, Katherine A. Comtois
A1: A behavioral economic perspective on adoption, implementation, and sustainment of evidence-based interventions
Lawrence A. Palinkas
A2: Towards making scale up of evidence-based practices in child welfare systems more efficient and affordable
Patricia Chamberlain
A3: Mixed method examination of strategic leadership for evidence-based practice implementation
Gregory A. Aarons, Amy E. Green, Mark. G. Ehrhart, Elise M. Trott, Cathleen E. Willging
A4: Implementing practice change in Federally Qualified Health Centers: Learning from leaders’ experiences
Maria E. Fernandez, Nicholas H. Woolf, Shuting (Lily) Liang, Natalia I. Heredia, Michelle Kegler, Betsy Risendal, Andrea Dwyer, Vicki Young, Dayna Campbell, Michelle Carvalho, Yvonne Kellar-Guenther
A3: Mixed method examination of strategic leadership for evidence-based practice implementation
Gregory A. Aarons, Amy E. Green, Mark. G. Ehrhart, Elise M. Trott, Cathleen E. Willging
A4: Implementing practice change in Federally Qualified Health Centers: Learning from leaders’ experiences
Maria E. Fernandez, Nicholas H. Woolf, Shuting (Lily) Liang, Natalia I. Heredia, Michelle Kegler, Betsy Risendal, Andrea Dwyer, Vicki Young, Dayna Campbell, Michelle Carvalho, Yvonne Kellar-Guenther
A5: Efficient synthesis: Using qualitative comparative analysis and the Consolidated Framework for Implementation Research across diverse studies
Laura J. Damschroder, Julie C. Lowery
A6: Establishing a veterans engagement group to empower patients and inform Veterans Affairs (VA) health services research
Sarah S. Ono, Kathleen F. Carlson, Erika K. Cottrell, Maya E. O’Neil, Travis L. Lovejoy
A7: Building patient-practitioner partnerships in community oncology settings to implement behavioral interventions for anxious and depressed cancer survivors
Joanna J. Arch, Jill L. Mitchell
A8: Tailoring a Cognitive Behavioral Therapy implementation protocol using mixed methods, conjoint analysis, and implementation teams
Cara C. Lewis, Brigid R. Marriott, Kelli Scott
A9: Wraparound Structured Assessment and Review (WrapSTAR): An efficient, yet comprehensive approach to Wraparound implementation evaluation
Jennifer Schurer Coldiron, Eric J. Bruns, Alyssa N. Hook
A10: Improving the efficiency of standardized patient assessment of clinician fidelity: A comparison of automated actor-based and manual clinician-based ratings
Benjamin C. Graham, Katelin Jordan
A11: Measuring fidelity on the cheap
Rochelle F. Hanson, Angela Moreland, Benjamin E. Saunders, Heidi S. Resnick
A12: Leveraging routine clinical materials to assess fidelity to an evidence-based psychotherapy
Shannon Wiltsey Stirman, Cassidy A. Gutner, Jennifer Gamarra, Dawne Vogt, Michael Suvak, Jennifer Schuster Wachen, Katherine Dondanville, Jeffrey S. Yarvis, Jim Mintz, Alan L. Peterson, Elisa V. Borah, Brett T. Litz, Alma Molino, Stacey Young McCaughanPatricia A. Resick
A13: The video vignette survey: An efficient process for gathering diverse community opinions to inform an intervention
Nancy Pandhi, Nora Jacobson, Neftali Serrano, Armando Hernandez, Elizabeth Zeidler- Schreiter, Natalie Wietfeldt, Zaher Karp
A14: Using integrated administrative data to evaluate implementation of a behavioral health and trauma screening for children and youth in foster care
Michael D. Pullmann, Barbara Lucenko, Bridget Pavelle, Jacqueline A. Uomoto, Andrea Negrete, Molly Cevasco, Suzanne E. U. Kerns
A15: Intermediary organizations as a vehicle to promote efficiency and speed of implementation
Robert P. Franks, Christopher Bory
A16: Applying the Consolidated Framework for Implementation Research constructs directly to qualitative data: The power of implementation science in action
Edward J. Miech, Teresa M. Damush
A17: Efficient and effective scaling-up, screening, brief interventions, and referrals to treatment (SBIRT) training: a snowball implementation model
Jason Satterfield, Derek Satre, Maria Wamsley, Patrick Yuan, Patricia O’Sullivan
A18: Matching models of implementation to system needs and capacities: addressing the human factor
Helen Best, Susan Velasquez
A19: Agency characteristics that facilitate efficient and successful implementation efforts
Miya Barnett, Lauren Brookman-Frazee, Jennifer Regan, Nicole Stadnick, Alison Hamilton, Anna Lau
A20: Rapid assessment process: Application to the Prevention and Early Intervention transformation in Los Angeles County
Jennifer Regan, Alison Hamilton, Nicole Stadnick, Miya Barnett, Anna Lau, Lauren Brookman-Frazee
A21: The development of the Evidence-Based Practice-Concordant Care Assessment: An assessment tool to examine treatment strategies across practices
Nicole Stadnick, Anna Lau, Miya Barnett, Jennifer Regan, Scott Roesch, Lauren Brookman-Frazee
A22: Refining a compilation of discrete implementation strategies and determining their importance and feasibility
Byron J. Powell, Thomas J. Waltz, Matthew J. Chinman, Laura Damschroder, Jeffrey L. Smith, Monica M. Matthieu, Enola K. Proctor, JoAnn E. Kirchner
A23: Structuring complex recommendations: Methods and general findings
Thomas J. Waltz, Byron J. Powell, Matthew J. Chinman, Laura J. Damschroder, Jeffrey L. Smith, Monica J. Matthieu, Enola K. Proctor, JoAnn E. Kirchner
A24: Implementing prolonged exposure for post-traumatic stress disorder in the Department of Veterans Affairs: Expert recommendations from the Expert Recommendations for Implementing Change (ERIC) project
Monica M. Matthieu, Craig S. Rosen, Thomas J. Waltz, Byron J. Powell, Matthew J. Chinman, Laura J. Damschroder, Jeffrey L. Smith, Enola K. Proctor, JoAnn E. Kirchner
A25: When readiness is a luxury: Co-designing a risk assessment and quality assurance process with violence prevention frontline workers in Seattle, WA
Sarah C. Walker, Asia S. Bishop, Mariko Lockhart
A26: Implementation potential of structured recidivism risk assessments with justice- involved veterans: Qualitative perspectives from providers
Allison L. Rodriguez, Luisa Manfredi, Andrea Nevedal, Joel Rosenthal, Daniel M. Blonigen
A27: Developing empirically informed readiness measures for providers and agencies for the Family Check-Up using a mixed methods approach
Anne M. Mauricio, Thomas D. Dishion, Jenna Rudo-Stern, Justin D. Smith
A28: Pebbles, rocks, and boulders: The implementation of a school-based social engagement intervention for children with autism
Jill Locke, Courtney Benjamin Wolk, Colleen Harker, Anne Olsen, Travis Shingledecker, Frances Barg, David Mandell, Rinad S. Beidas
A29: Problem Solving Teletherapy (PST.Net): A stakeholder analysis examining the feasibility and acceptability of teletherapy in community based aging services
Marissa C. Hansen, Maria P. Aranda, Isabel Torres-Vigil
A30: A case of collaborative intervention design eventuating in behavior therapy sustainment and diffusion
Bryan Hartzler
A31: Implementation of suicide risk prevention in an integrated delivery system: Mental health specialty services
Bradley Steinfeld, Tory Gildred, Zandrea Harlin, Fredric Shephard
A32: Implementation team, checklist, evaluation, and feedback (ICED): A step-by-step approach to Dialectical Behavior Therapy program implementation
Matthew S. Ditty, Andrea Doyle, John A. Bickel III, Katharine Cristaudo
A33: The challenges in implementing muliple evidence-based practices in a community mental health setting
Dan Fox, Sonia Combs
A34: Using electronic health record technology to promote and support evidence-based practice assessment and treatment intervention
David H. Lischner
A35: Are existing frameworks adequate for measuring implementation outcomes? Results from a new simulation methodology
Richard A. Van Dorn, Stephen J. Tueller, Jesse M. Hinde, Georgia T. Karuntzos
A36: Taking global local: Evaluating training of Washington State clinicians in a modularized cogntive behavioral therapy approach designed for low-resource settings
Maria Monroe-DeVita, Roselyn Peterson, Doyanne Darnell, Lucy Berliner, Shannon Dorsey, Laura K. Murray
A37: Attitudes toward evidence-based practices across therapeutic orientations
Yevgeny Botanov, Beverly Kikuta, Tianying Chen, Marivi Navarro-Haro, Anthony DuBose, Kathryn E. Korslund, Marsha M. Linehan
A38: Predicting the use of an evidence-based intervention for autism in birth-to-three programs
Colleen M. Harker, Elizabeth A. Karp, Sarah R. Edmunds, Lisa V. Ibañez, Wendy L. Stone
A39: Supervision practices and improved fidelity across evidence-based practices: A literature review
Mimi Choy-Brown
A40: Beyond symptom tracking: clinician perceptions of a hybrid measurement feedback system for monitoring treatment fidelity and client progress
Jack H. Andrews, Benjamin D. Johnides, Estee M. Hausman, Kristin M. Hawley
A41: A guideline decision support tool: From creation to implementation
Beth Prusaczyk, Alex Ramsey, Ana Baumann, Graham Colditz, Enola K. Proctor
A42: Dabblers, bedazzlers, or total makeovers: Clinician modification of a common elements cognitive behavioral therapy approach
Rosemary D. Meza, Shannon Dorsey, Shannon Wiltsey-Stirman, Georganna Sedlar, Leah Lucid
A43: Characterization of context and its role in implementation: The impact of structure, infrastructure, and metastructure
Caitlin Dorsey, Brigid Marriott, Nelson Zounlome, Cara Lewis
A44: Effects of consultation method on implementation of cognitive processing therapy for post-traumatic stress disorder
Cassidy A. Gutner, Candice M. Monson, Norman Shields, Marta Mastlej, Meredith SH Landy, Jeanine Lane, Shannon Wiltsey Stirman
A45: Cross-validation of the Implementation Leadership Scale factor structure in child welfare service organizations
Natalie K. Finn, Elisa M. Torres, Mark. G. Ehrhart, Gregory A. Aarons
A46: Sustainability of integrated smoking cessation care in Veterans Affairs posttraumatic stress disorder clinics: A qualitative analysis of focus group data from learning collaborative participants
Carol A. Malte, Aline Lott, Andrew J. Saxon
A47: Key characteristics of effective mental health trainers: The creation of the Measure of Effective Attributes of Trainers (MEAT)
Meredith Boyd, Kelli Scott, Cara C. Lewis
A48: Coaching to improve teacher implementation of evidence-based practices (EBPs)
Jennifer D. Pierce
A49: Factors influencing the implementation of peer-led health promotion programs targeting seniors: A literature review
Agathe Lorthios-Guilledroit, Lucie Richard, Johanne Filiatrault
A50: Developing treatment fidelity rating systems for psychotherapy research: Recommendations and lessons learned
Kevin Hallgren, Shirley Crotwell, Rosa Muñoz, Becky Gius, Benjamin Ladd, Barbara McCrady, Elizabeth Epstein
A51: Rapid translation of alcohol prevention science
John D. Clapp, Danielle E. Ruderman
A52: Factors implicated in successful implementation: evidence to inform improved implementation from high and low-income countries
Melanie Barwick, Raluca Barac, Stanley Zlotkin, Laila Salim, Marnie
Davidson
A53: Tracking implementation strategies prospectively: A practical approach
Alicia C. Bunger, Byron J. Powell, Hillary A. Robertson
A54: Trained but not implementing: the need for effective implementation planning tools
Christopher Botsko
A55: Evidence, context, and facilitation variables related to implementation of Dialectical Behavior Therapy: Qualitative results from a mixed methods inquiry in the Department of Veterans Affairs
Sara J. Landes, Brandy N. Smith, Allison L. Rodriguez, Lindsay R. Trent, Monica M. Matthieu
A56: Learning from implementation as usual in children’s mental health
Byron J. Powell, Enola K. Proctor
A57: Rates and predictors of implementation after Dialectical Behavior Therapy Intensive Training
Melanie S. Harned, Marivi Navarro-Haro, Kathryn E. Korslund, Tianying Chen, Anthony DuBose, André Ivanoff, Marsha M. Linehan
A58: Socio-contextual determinants of research evidence use in public-youth systems of care
Antonio R. Garcia, Minseop Kim, Lawrence A. Palinkas, Lonnie Snowden, John Landsverk
A59: Community resource mapping to integrate evidence-based depression treatment in primary care in Brazil: A pilot project
Annika C. Sweetland, Maria Jose Fernandes, Edilson Santos, Cristiane Duarte, Afrânio Kritski, Noa Krawczyk, Caitlin Nelligan, Milton L. Wainberg
A60: The use of concept mapping to efficiently identify determinants of implementation in the National Institute of Health--President’s Emergent Plan for AIDS Relief Prevention of Mother to Child HIV Transmission Implementation Science Alliance
Gregory A. Aarons, David H. Sommerfeld, Benjamin Chi, Echezona Ezeanolue, Rachel Sturke, Lydia Kline, Laura Guay, George Siberry
A61: Longitudinal remote consultation for implementing collaborative care for depression
Ian M. Bennett, Rinad Beidas, Rachel Gold, Johnny Mao, Diane Powers, Mindy Vredevoogd, Jurgen Unutzer
A62: Integrating a peer coach model to support program implementation and ensure long- term sustainability of the Incredible Years in community-based settings
Jennifer Schroeder, Lane Volpe, Julie Steffen
A63: Efficient sustainability: Existing community based supervisors as evidence-based treatment supports
Shannon Dorsey, Michael D Pullmann, Suzanne E. U. Kerns, Nathaniel Jungbluth, Lucy Berliner, Kelly Thompson, Eliza Segell
A64: Establishment of a national practice-based implementation network to accelerate adoption of evidence-based and best practices
Pearl McGee-Vincent, Nancy Liu, Robyn Walser, Jennifer Runnals, R. Keith Shaw, Sara J. Landes, Craig Rosen, Janet Schmidt, Patrick Calhoun
A65: Facilitation as a mechanism of implementation in a practice-based implementation network: Improving care in a Department of Veterans Affairs post-traumatic stress disorder outpatient clinic
Ruth L. Varkovitzky, Sara J. Landes
A66: The ACT SMART Toolkit: An implementation strategy for community-based organizations providing services to children with autism spectrum disorder
Amy Drahota, Jonathan I. Martinez, Brigitte Brikho, Rosemary Meza, Aubyn C. Stahmer, Gregory A. Aarons
A67: Supporting Policy In Health with Research: An intervention trial (SPIRIT) - protocol and early findings
Anna Williamson
A68: From evidence based practice initiatives to infrastructure: Lessons learned from a public behavioral health system’s efforts to promote evidence based practices
Ronnie M. Rubin, Byron J. Powell, Matthew O. Hurford, Shawna L. Weaver, Rinad S. Beidas, David S. Mandell, Arthur C. Evans
A69: Applying the policy ecology model to Philadelphia’s behavioral health transformation efforts
Byron J. Powell, Rinad S. Beidas, Ronnie M. Rubin, Rebecca E. Stewart, Courtney Benjamin Wolk, Samantha L. Matlin, Shawna Weaver, Matthew O. Hurford, Arthur C. Evans, Trevor R. Hadley, David S. Mandell
A70: A model for providing methodological expertise to advance dissemination and implementation of health discoveries in Clinical and Translational Science Award institutions
Donald R. Gerke, Beth Prusaczyk, Ana Baumann, Ericka M. Lewis, Enola K. Proctor
A71: Establishing a research agenda for the Triple P Implementation Framework
Jenna McWilliam, Jacquie Brown, Michelle Tucker
A72: Cheap and fast, but what is “best?”: Examining implementation outcomes across sites in a state-wide scaled-up evidence-based walking program, Walk With Ease
Kathleen P Conte
A73: Measurement feedback systems in mental health: Initial review of capabilities and characteristics
Aaron R. Lyon, Meredith Boyd, Abigail Melvin, Cara C. Lewis, Freda Liu, Nathaniel Jungbluth
A74: A qualitative investigation of case managers’ attitudes toward implementation of a measurement feedback system in a public mental health system for youth
Amelia Kotte, Kaitlin A. Hill, Albert C. Mah, Priya A. Korathu-Larson, Janelle R. Au, Sonia Izmirian, Scott Keir, Brad J. Nakamura, Charmaine K. Higa-McMillan
A75: Multiple pathways to sustainability: Using Qualitative Comparative Analysis to uncover the necessary and sufficient conditions for successful community-based implementation
Brittany Rhoades Cooper, Angie Funaiole, Eleanor Dizon
A76: Prescribers’ perspectives on opioids and benzodiazepines and medication alerts to reduce co-prescribing of these medications
Eric J. Hawkins, Carol A. Malte, Hildi J. Hagedorn, Douglas Berger, Anissa Frank, Aline Lott, Carol E. Achtmeyer, Anthony J. Mariano, Andrew J. Saxon
A77: Adaptation of Coordinated Anxiety Learning and Management for comorbid anxiety and substance use disorders: Delivery of evidence-based treatment for anxiety in addictions treatment centers
Kate Wolitzky-Taylor, Richard Rawson, Richard Ries, Peter Roy-Byrne, Michelle Craske
A78: Opportunities and challenges of measuring program implementation with online surveys
Dena Simmons, Catalina Torrente, Lori Nathanson, Grace Carroll
A79: Observational assessment of fidelity to a family-centered prevention program: Effectiveness and efficiency
Justin D. Smith, Kimbree Brown, Karina Ramos, Nicole Thornton, Thomas J. Dishion, Elizabeth A. Stormshak, Daniel S. Shaw, Melvin N. Wilson
A80: Strategies and challenges in housing first fidelity: A multistate qualitative analysis
Mimi Choy-Brown, Emmy Tiderington, Bikki Tran Smith, Deborah K. Padgett
A81: Procurement and contracting as an implementation strategy: Getting To Outcomes® contracting
Ronnie M. Rubin, Marilyn L. Ray, Abraham Wandersman, Andrea Lamont, Gordon Hannah, Kassandra A. Alia, Matthew O. Hurford, Arthur C. Evans
A82: Web-based feedback to aid successful implementation: The interactive Stages of Implementation Completion (SIC)TM tool
Lisa Saldana, Holle Schaper, Mark Campbell, Patricia Chamberlain
A83: Efficient methodologies for monitoring fidelity in routine implementation: Lessons from the Allentown Social Emotional Learning Initiative
Valerie B. Shapiro, B.K. Elizabeth Kim, Jennifer L. Fleming, Paul A. LeBuffe
A84: The Society for Implementation Research Collaboration (SIRC) implementation development workshop: Results from a new methodology for enhancing implementation science proposals
Sara J. Landes, Cara C. Lewis, Allison L. Rodriguez, Brigid R. Marriott, Katherine Anne Comtois
A85: An update on the Society for Implementation Research Collaboration (SIRC) Instrument Review Project
doi:10.1186/s13012-016-0428-0
PMCID: PMC4928139  PMID: 27357964
25.  Provider management strategies of abnormal test result alerts: a cognitive task analysis 
Objective
Electronic medical records (EMRs) facilitate abnormal test result communication through “alert” notifications. The aim was to evaluate how primary care providers (PCPs) manage alerts related to critical diagnostic test results on their EMR screens, and compare alert-management strategies of providers with high versus low rates of timely follow-up of results.
Design
28 PCPs from a large, tertiary care Veterans Affairs Medical Center (VAMC) were purposively sampled according to their rates of timely follow-up of alerts, determined in a previous study. Using techniques from cognitive task analysis, participants were interviewed about how and when they manage alerts, focusing on four alert-management features to filter, sort and reduce unnecessary alerts on their EMR screens.
Results
Provider knowledge of alert-management features ranged between 4% and 75%. Almost half (46%) of providers did not use any of these features, and none used more than two. Providers with higher versus lower rates of timely follow-up used the four features similarly, except one (customizing alert notifications). Providers with low rates of timely follow-up tended to manually scan the alert list and process alerts heuristically using their clinical judgment. Additionally, 46% of providers used at least one workaround strategy to manage alerts.
Conclusion
Considerable heterogeneity exists in provider use of alert-management strategies; specific strategies may be associated with lower rates of timely follow-up. Standardization of alert-management strategies including improving provider knowledge of appropriate tools in the EMR to manage alerts could reduce the lack of timely follow-up of abnormal diagnostic test results.
doi:10.1197/jamia.M3200
PMCID: PMC2995633  PMID: 20064805
medical records systems; computerized; task performance and analysis; diagnostic errors/classification; primary healthcare; software

Results 1-25 (1440621)