PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (571257)

Clipboard (0)
None

Related Articles

1.  Development and Evaluation of a Checklist for Medication Order Review by Pharmacists 
Background:
To create a checklist of the tasks that a pharmacist must perform during medication order review in the hospital setting and to evaluate the utility of and pharmacists’ satisfaction with the checklist.
Methods:
An evidence-based checklist for medication order review was developed, with items related to order urgency, verification of patients’ identity, therapeutic review, and actionable items. Pharmacists were educated about the checklist, and it was made available at 2 community hospitals in an urban setting. Pharmacists completed a nonvalidated satisfaction survey and participated in focus groups or interviews within 3 months after implementation of the checklist. Qualitative descriptive theory was used to identify themes within the data. Near-miss occurrence reports for the 3 months before and after implementation of the checklist were quantified.
Results:
Of 16 pharmacists who were involved in the implementation phase, 14 participated in focus groups or an interview, and 11 responded to the survey. All respondents felt that the primary role of the checklist was for training. They felt that the checklist could be useful when reviewing high-alert or unfamiliar medications or therapy for patients with complex medications. The checklist was most helpful when it was used as a reminder, on an as-needed basis. Nine (82%) of the 11 survey respondents indicated that the checklist standardized the process of medication order review, the same number felt that it prevented accidental omission of critical checks, and 8 (73%) felt that it improved patient safety. Education was necessary to reinforce the purpose of the checklist and its self-check nature. There was no difference in the number of near misses in the pharmacy between the 3-month periods before and after implementation of the checklist.
Conclusion:
Pharmacists participating in the study felt that a checklist for medication order review had a role in training new pharmacists and standardizing processes.
PMCID: PMC3130755  PMID: 22479055
checklist; pharmacist; medication order review; liste de contrôle; pharmacien; validation des ordonnances de médicaments
2.  Multiple Intravenous Infusions Phase 1b 
Background
Minimal research has been conducted into the potential patient safety issues related to administering multiple intravenous (IV) infusions to a single patient. Previous research has highlighted that there are a number of related safety risks. In Phase 1a of this study, an analysis of 2 national incident-reporting databases (Institute for Safe Medical Practices Canada and United States Food and Drug Administration MAUDE) found that a high percentage of incidents associated with the administration of multiple IV infusions resulted in patient harm.
Objectives
The primary objectives of Phase 1b of this study were to identify safety issues with the potential to cause patient harm stemming from the administration of multiple IV infusions; and to identify how nurses are being educated on key principles required to safely administer multiple IV infusions.
Data Sources and Review Methods
A field study was conducted at 12 hospital clinical units (sites) across Ontario, and telephone interviews were conducted with program coordinators or instructors from both the Ontario baccalaureate nursing degree programs and the Ontario postgraduate Critical Care Nursing Certificate programs. Data were analyzed using Rasmussen’s 1997 Risk Management Framework and a Health Care Failure Modes and Effects Analysis.
Results
Twenty-two primary patient safety issues were identified with the potential to directly cause patient harm. Seventeen of these (critical issues) were categorized into 6 themes. A cause-consequence tree was established to outline all possible contributing factors for each critical issue. Clinical recommendations were identified for immediate distribution to, and implementation by, Ontario hospitals. Future investigation efforts were planned for Phase 2 of the study.
Limitations
This exploratory field study identifies the potential for errors, but does not describe the direct observation of such errors, except in a few cases where errors were observed. Not all issues are known in advance, and the frequency of errors is too low to be observed in the time allotted and with the limited sample of observations.
Conclusions
The administration of multiple IV infusions to a single patient is a complex task with many potential associated patient safety risks. Improvements to infusion and infusion-related technology, education standards, clinical best practice guidelines, hospital policies, and unit work practices are required to reduce the risk potential. This report makes several recommendations to Ontario hospitals so that they can develop an awareness of the issues highlighted in this report and minimize some of the risks. Further investigation of mitigating strategies is required and will be undertaken in Phase 2 of this research.
Plain Language Summary
Patients, particularly in critical care environments, often require multiple intravenous (IV) medications via large volumetric or syringe infusion pumps. The infusion of multiple IV medications is not without risk; unintended errors during these complex procedures have resulted in patient harm. However, the range of associated risks and the factors contributing to these risks are not well understood.
Health Quality Ontario’s Ontario Health Technology Advisory Committee commissioned the Health Technology Safety Research Team at the University Health Network to conduct a multi-phase study to identify and mitigate the risks associated with multiple IV infusions. Some of the questions addressed by the team were as follows: What is needed to reduce the risk of errors for individuals who are receiving a lot of medications? What strategies work best?
The initial report, Multiple Intravenous Infusions Phase 1a: Situation Scan Summary Report, summarizes the interim findings based on a literature review, an incident database review, and a technology scan.
The Health Technology Safety Research Team worked in close collaboration with the Institute for Safe Medication Practices Canada on an exploratory study to understand the risks associated with multiple IV infusions and the degree to which nurses are educated to help mitigate them. The current report, Multiple Intravenous Infusions Phase 1b: Practice and Training Scan, presents the findings of a field study of 12 hospital clinical units across Ontario, as well as 13 interviews with educators from baccalaureate-level nursing degree programs and postgraduate Critical Care Nursing Certificate programs. It makes 9 recommendations that emphasize best practices for the administration of multiple IV infusions and pertain to secondary infusions, line identification, line set-up and removal, and administering IV bolus medications.
The Health Technology Safety Research Team has also produced an associated report for hospitals entitled Mitigating the Risks Associated With Multiple IV Infusions: Recommendations Based on a Field Study of Twelve Ontario Hospitals, which highlights the 9 interim recommendations and provides a brief rationale for each one.
PMCID: PMC3377572  PMID: 23074426
3.  Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety 
Long working hours and sleep deprivation have been a facet of physician training in the US since the advent of the modern residency system. However, the scientific evidence linking fatigue with deficits in human performance, accidents and errors in industries from aeronautics to medicine, nuclear power, and transportation has mounted over the last 40 years. This evidence has also spawned regulations to help ensure public safety across safety-sensitive industries, with the notable exception of medicine.
In late 2007, at the behest of the US Congress, the Institute of Medicine embarked on a year-long examination of the scientific evidence linking resident physician sleep deprivation with clinical performance deficits and medical errors. The Institute of Medicine’s report, entitled “Resident duty hours: Enhancing sleep, supervision and safety”, published in January 2009, recommended new limits on resident physician work hours and workload, increased supervision, a heightened focus on resident physician safety, training in structured handovers and quality improvement, more rigorous external oversight of work hours and other aspects of residency training, and the identification of expanded funding sources necessary to implement the recommended reforms successfully and protect the public and resident physicians themselves from preventable harm.
Given that resident physicians comprise almost a quarter of all physicians who work in hospitals, and that taxpayers, through Medicare and Medicaid, fund graduate medical education, the public has a deep investment in physician training. Patients expect to receive safe, high-quality care in the nation’s teaching hospitals. Because it is their safety that is at issue, their voices should be central in policy decisions affecting patient safety. It is likewise important to integrate the perspectives of resident physicians, policy makers, and other constituencies in designing new policies. However, since its release, discussion of the Institute of Medicine report has been largely confined to the medical education community, led by the Accreditation Council for Graduate Medical Education (ACGME).
To begin gathering these perspectives and developing a plan to implement safer work hours for resident physicians, a conference entitled “Enhancing sleep, supervision and safety: What will it take to implement the Institute of Medicine recommendations?” was held at Harvard Medical School on June 17–18, 2010. This White Paper is a product of a diverse group of 26 representative stakeholders bringing relevant new information and innovative practices to bear on a critical patient safety problem. Given that our conference included experts from across disciplines with diverse perspectives and interests, not every recommendation was endorsed by each invited conference participant. However, every recommendation made here was endorsed by the majority of the group, and many were endorsed unanimously. Conference members participated in the process, reviewed the final product, and provided input before publication. Participants provided their individual perspectives, which do not necessarily represent the formal views of any organization.
In September 2010 the ACGME issued new rules to go into effect on July 1, 2011. Unfortunately, they stop considerably short of the Institute of Medicine’s recommendations and those endorsed by this conference. In particular, the ACGME only applied the limitation of 16 hours to first-year resident physicans. Thus, it is clear that policymakers, hospital administrators, and residency program directors who wish to implement safer health care systems must go far beyond what the ACGME will require. We hope this White Paper will serve as a guide and provide encouragement for that effort.
Resident physician workload and supervision
By the end of training, a resident physician should be able to practice independently. Yet much of resident physicians’ time is dominated by tasks with little educational value. The caseload can be so great that inadequate reflective time is left for learning based on clinical experiences. In addition, supervision is often vaguely defined and discontinuous. Medical malpractice data indicate that resident physicians are frequently named in lawsuits, most often for lack of supervision. The recommendations are: The ACGME should adjust resident physicians workload requirements to optimize educational value. Resident physicians as well as faculty should be involved in work redesign that eliminates nonessential and noneducational activity from resident physician dutiesMechanisms should be developed for identifying in real time when a resident physician’s workload is excessive, and processes developed to activate additional providersTeamwork should be actively encouraged in delivery of patient care. Historically, much of medical training has focused on individual knowledge, skills, and responsibility. As health care delivery has become more complex, it will be essential to train resident and attending physicians in effective teamwork that emphasizes collective responsibility for patient care and recognizes the signs, both individual and systemic, of a schedule and working conditions that are too demanding to be safeHospitals should embrace the opportunities that resident physician training redesign offers. Hospitals should recognize and act on the potential benefits of work redesign, eg, increased efficiency, reduced costs, improved quality of care, and resident physician and attending job satisfactionAttending physicians should supervise all hospital admissions. Resident physicians should directly discuss all admissions with attending physicians. Attending physicians should be both cognizant of and have input into the care patients are to receive upon admission to the hospitalInhouse supervision should be required for all critical care services, including emergency rooms, intensive care units, and trauma services. Resident physicians should not be left unsupervised to care for critically ill patients. In settings in which the acuity is high, physicians who have completed residency should provide direct supervision for resident physicians. Supervising physicians should always be physically in the hospital for supervision of resident physicians who care for critically ill patientsThe ACGME should explicitly define “good” supervision by specialty and by year of training. Explicit requirements for intensity and level of training for supervision of specific clinical scenarios should be providedCenters for Medicare and Medicaid Services (CMS) should use graduate medical education funding to provide incentives to programs with proven, effective levels of supervision. Although this action would require federal legislation, reimbursement rules would help to ensure that hospitals pay attention to the importance of good supervision and require it from their training programs
Resident physician work hours
Although the IOM “Sleep, supervision and safety” report provides a comprehensive review and discussion of all aspects of graduate medical education training, the report’s focal point is its recommendations regarding the hours that resident physicians are currently required to work. A considerable body of scientific evidence, much of it cited by the Institute of Medicine report, describes deteriorating performance in fatigued humans, as well as specific studies on resident physician fatigue and preventable medical errors.
The question before this conference was what work redesign and cultural changes are needed to reform work hours as recommended by the Institute of Medicine’s evidence-based report? Extensive scientific data demonstrate that shifts exceeding 12–16 hours without sleep are unsafe. Several principles should be followed in efforts to reduce consecutive hours below this level and achieve safer work schedules. The recommendations are: Limit resident physician work hours to 12–16 hour maximum shiftsA minimum of 10 hours off duty should be scheduled between shiftsResident physician input into work redesign should be actively solicitedSchedules should be designed that adhere to principles of sleep and circadian science; this includes careful consideration of the effects of multiple consecutive night shifts, and provision of adequate time off after night work, as specified in the IOM reportResident physicians should not be scheduled up to the maximum permissible limits; emergencies frequently occur that require resident physicians to stay longer than their scheduled shifts, and this should be anticipated in scheduling resident physicians’ work shiftsHospitals should anticipate the need for iterative improvement as new schedules are initiated; be prepared to learn from the initial phase-in, and change the plan as neededAs resident physician work hours are redesigned, attending physicians should also be considered; a potential consequence of resident physician work hour reduction and increased supervisory requirements may be an increase in work for attending physicians; this should be carefully monitored, and adjustments to attending physician work schedules made as needed to prevent unsafe work hours or working conditions for this group“Home call” should be brought under the overall limits of working hours; work load and hours should be monitored in each residency program to ensure that resident physicians and fellows on home call are getting sufficient sleepMedicare funding for graduate medical education in each hospital should be linked with adherence to the Institute of Medicine limits on resident physician work hours
Moonlighting by resident physicians
The Institute of Medicine report recommended including external as well as internal moonlighting in working hour limits. The recommendation is: All moonlighting work hours should be included in the ACGME working hour limits and actively monitored. Hospitals should formalize a moonlighting policy and establish systems for actively monitoring resident physician moonlighting
Safety of resident physicians
The “Sleep, supervision and safety” report also addresses fatigue-related harm done to resident physicians themselves. The report focuses on two main sources of physical injury to resident physicians impaired by fatigue, ie, needle-stick exposure to blood-borne pathogens and motor vehicle crashes. Providing safe transportation home for resident physicians is a logistical and financial challenge for hospitals. Educating physicians at all levels on the dangers of fatigue is clearly required to change driving behavior so that safe hospital-funded transport home is used effectively. Fatigue-related injury prevention (including not driving while drowsy) should be taught in medical school and during residency, and reinforced with attending physicians; hospitals and residency programs must be informed that resident physicians’ ability to judge their own level of impairment is impaired when they are sleep deprived; hence, leaving decisions about the capacity to drive to impaired resident physicians is not recommendedHospitals should provide transportation to all resident physicians who report feeling too tired to drive safely; in addition, although consecutive work should not exceed 16 hours, hospitals should provide transportation for all resident physicians who, because of unforeseen reasons or emergencies, work for longer than consecutive 24 hours; transportation under these circumstances should be automatically provided to house staff, and should not rely on self-identification or request
Training in effective handovers and quality improvement
Handover practice for resident physicians, attendings, and other health care providers has long been identified as a weak link in patient safety throughout health care settings. Policies to improve handovers of care must be tailored to fit the appropriate clinical scenario, recognizing that information overload can also be a problem. At the heart of improving handovers is the organizational effort to improve quality, an effort in which resident physicians have typically been insufficiently engaged. The recommendations are: Hospitals should train attending and resident physicians in effective handovers of careHospitals should create uniform processes for handovers that are tailored to meet each clinical setting; all handovers should be done verbally and face-to-face, but should also utilize written toolsWhen possible, hospitals should integrate hand-over tools into their electronic medical records (EMR) systems; these systems should be standardized to the extent possible across residency programs in a hospital, but may be tailored to the needs of specific programs and services; federal government should help subsidize adoption of electronic medical records by hospitals to improve signoutWhen feasible, handovers should be a team effort including nurses, patients, and familiesHospitals should include residents in their quality improvement and patient safety efforts; the ACGME should specify in their core competency requirements that resident physicians work on quality improvement projects; likewise, the Joint Commission should require that resident physicians be included in quality improvement and patient safety programs at teaching hospitals; hospital administrators and residency program directors should create opportunities for resident physicians to become involved in ongoing quality improvement projects and root cause analysis teams; feedback on successful quality improvement interventions should be shared with resident physicians and broadly disseminatedQuality improvement/patient safety concepts should be integral to the medical school curriculum; medical school deans should elevate the topics of patient safety, quality improvement, and teamwork; these concepts should be integrated throughout the medical school curriculum and reinforced throughout residency; mastery of these concepts by medical students should be tested on the United States Medical Licensing Examination (USMLE) stepsFederal government should support involvement of resident physicians in quality improvement efforts; initiatives to improve quality by including resident physicians in quality improvement projects should be financially supported by the Department of Health and Human Services
Monitoring and oversight of the ACGME
While the ACGME is a key stakeholder in residency training, external voices are essential to ensure that public interests are heard in the development and monitoring of standards. Consequently, the Institute of Medicine report recommended external oversight and monitoring through the Joint Commission and Centers for Medicare and Medicaid Services (CMS). The recommendations are: Make comprehensive fatigue management a Joint Commission National Patient Safety Goal; fatigue is a safety concern not only for resident physicians, but also for nurses, attending physicians, and other health care workers; the Joint Commission should seek to ensure that all health care workers, not just resident physicians, are working as safely as possibleFederal government, including the Centers for Medicare and Medicaid Services and the Agency for Healthcare Research and Quality, should encourage development of comprehensive fatigue management programs which all health systems would eventually be required to implementMake ACGME compliance with working hours a “ condition of participation” for reimbursement of direct and indirect graduate medical education costs; financial incentives will greatly increase the adoption of and compliance with ACGME standards
Future financial support for implementation
The Institute of Medicine’s report estimates that $1.7 billion (in 2008 dollars) would be needed to implement its recommendations. Twenty-five percent of that amount ($376 million) will be required just to bring hospitals into compliance with the existing 2003 ACGME rules. Downstream savings to the health care system could potentially result from safer care, but these benefits typically do not accrue to hospitals and residency programs, who have been asked historically to bear the burden of residency reform costs. The recommendations are: The Institute of Medicine should convene a panel of stakeholders, including private and public funders of health care and graduate medical education, to lay down the concrete steps necessary to identify and allocate the resources needed to implement the recommendations contained in the IOM “Resident duty hours: Enhancing sleep, supervision and safety” report. Conference participants suggested several approaches to engage public and private support for this initiativeEfforts to find additional funding to implement the Institute of Medicine recommendations should focus more broadly on patient safety and health care delivery reform; policy efforts focused narrowly upon resident physician work hours are less likely to succeed than broad patient safety initiatives that include residency redesign as a key componentHospitals should view the Institute of Medicine recommendations as an opportunity to begin resident physician work redesign projects as the core of a business model that embraces safety and ultimately saves resourcesBoth the Secretary of Health and Human Services and the Director of the Centers for Medicare and Medicaid Services should take the Institute of Medicine recommendations into consideration when promulgating rules for innovation grantsThe National Health Care Workforce Commission should consider the Institute of Medicine recommendations when analyzing the nation’s physician workforce needs
Recommendations for future research
Conference participants concurred that convening the stakeholders and agreeing on a research agenda was key. Some observed that some sectors within the medical education community have been reluctant to act on the data. Several logical funders for future research were identified. But above all agencies, Centers for Medicare and Medicaid Services is the only stakeholder that funds graduate medical education upstream and will reap savings downstream if preventable medical errors are reduced as a result of reform of resident physician work hours.
doi:10.2147/NSS.S19649
PMCID: PMC3630963  PMID: 23616719
resident; hospital; working hours; safety
4.  Psychometric Properties of Self-Report Concussion Scales and Checklists 
Journal of Athletic Training  2012;47(2):221-223.
Reference/Citation:
Alla S, Sullivan SJ, Hale L, McCrory P. Self-report scales/checklists for the measurement of concussion symptoms: a systematic review. Br J Sports Med. 2009;43 (suppl 1):i3–i12.
Clinical Question:
Which self-report symptom scales or checklists are psychometrically sound for clinical use to assess sport-related concussion?
Data Sources:
Articles available in full text, published from the establishment of each database through December 2008, were identified from PubMed, Medline, CINAHL, Scopus, Web of Science, SPORTDiscus, PsycINFO, and AMED. Search terms included brain concussion, signs or symptoms, and athletic injuries, in combination with the AND Boolean operator, and were limited to studies published in English. The authors also hand searched the reference lists of retrieved articles. Additional searches of books, conference proceedings, theses, and Web sites of commercial scales were done to provide additional information about the psychometric properties and development for those scales when needed in articles meeting the inclusion criteria.
Study Selection:
Articles were included if they identified all the items on the scale and the article was either an original research report describing the use of scales in the evaluation of concussion symptoms or a review article that discussed the use or development of concussion symptom scales. Only articles published in English and available in full text were included.
Data Extraction:
From each study, the following information was extracted by the primary author using a standardized protocol: study design, publication year, participant characteristics, reliability of the scale, and details of the scale or checklist, including name, number of items, time of measurement, format, mode of report, data analysis, scoring, and psychometric properties. A quality assessment of included studies was done using 16 items from the Downs and Black checklist1 and assessed reporting, internal validity, and external validity.
Main Results:
The initial database search identified 421 articles. After 131 duplicate articles were removed, 290 articles remained and were added to 17 articles found during the hand search, for a total of 307 articles; of those, 295 were available in full text. Sixty articles met the inclusion criteria and were used in the systematic review. The quality of the included studies ranged from 9 to 15 points out of a maximum quality score of 17. The included articles were published between 1995 and 2008 and included a collective total of 5864 concussed athletes and 5032 nonconcussed controls, most of whom participated in American football. The majority of the studies were descriptive studies monitoring the resolution of concussive self-report symptoms compared with either a preseason baseline or healthy control group, with a smaller number of studies (n = 8) investigating the development of a scale.
The authors initially identified 20 scales that were used among the 60 included articles. Further review revealed that 14 scales were variations of the Pittsburgh Steelers postconcussion scale (the Post-Concussion Scale, Post-Concussion Scale: Revised, Post-Concussion Scale: ImPACT, Post-Concussion Symptom Scale: Vienna, Graded Symptom Checklist [GSC], Head Injury Scale, McGill ACE Post-Concussion Symptoms Scale, and CogState Sport Symptom Checklist), narrowing down to 6 core scales, which the authors discussed further. The 6 core scales were the Pittsburgh Steelers Post-Concussion Scale (17 items), Post-Concussion Symptom Assessment Questionnaire (10 items), Concussion Resolution Index postconcussion questionnaire (15 items), Signs and Symptoms Checklist (34 items), Sport Concussion Assessment Tool (SCAT) postconcussion symptom scale (25 items), and Concussion Symptom Inventory (12 items). Each of the 6 core scales includes symptoms associated with sport-related concussion; however, the number of items on each scale varied. A 7-point Likert scale was used on most scales, with a smaller number using a dichotomous (yes/no) classification.
Only 7 of the 20 scales had published psychometric properties, and only 1 scale, the Concussion Symptom Inventory, was empirically driven (Rasch analysis), with development of the scale occurring before its clinical use. Internal consistency (Cronbach α) was reported for the Post-Concussion Scale (.87), Post-Concussion Scale: ImPACT 22-item (.88–.94), Head Injury Scale 9-item (.78), and Head Injury Scale 16-item (.84). Test-retest reliability has been reported only for the Post-Concussion Scale (Spearman r = .55) and the Post-Concussion Scale: ImPACT 21-item (Pearson r = .65). With respect to validity, the SCAT postconcussion scale has demonstrated face and content validity, the Post-Concussion Scale: ImPACT 22-item and Head Injury Scale 9-item have reported construct validity, and the Head Injury Scale 9-item and 16-item have published factorial validity.
Sensitivity and specificity have been reported only with the GSC (0.89 and 1.0, respectively) and the Post-Concussion Scale: ImPACT 21-item when combined with the neurocognitive component of ImPACT (0.819 and 0.849, respectively). Meaningful change scores were reported for the Post-Concussion Scale (14.8 points), Post-Concussion Scale: ImPACT 22-item (6.8 points), and Post-Concussion Scale: ImPACT 21-item (standard error of the difference = 7.17; 80% confidence interval = 9.18).
Conclusions:
Numerous scales exist for measuring the number and severity of concussion-related symptoms, with most evolving from the neuropsychology literature pertaining to head-injured populations. However, very few of these were created in a systematic manner that follows scale development processes and have published psychometric properties. Clinicians need to understand these limitations when choosing and using a symptom scale for inclusion in a concussion assessment battery. Future authors should assess the underlying constructs and measurement properties of currently available scales and use the ever-increasing prospective data pools of concussed athlete information to develop scales following appropriate, systematic processes.
PMCID: PMC3418135  PMID: 22488289
mild traumatic brain injuries; evaluation; reliability; validity; sensitivity; specificity
5.  Validation of a Detailed Scoring Checklist for Use During Advanced Cardiac Life Support Certification 
Introduction
Defining valid, reliable, defensible, and generalizable standards for the evaluation of learner performance is a key issue in assessing both baseline competence and mastery in medical education. However, prior to setting these standards of performance, the reliability of the scores yielding from a grading tool must be assessed. Accordingly, the purpose of this study was to assess the reliability of scores generated from a set of grading checklists used by non-expert raters during simulations of American Heart Association (AHA) MegaCodes.
Methods
The reliability of scores generated from a detailed set of checklists, when used by four non-expert raters, was tested by grading team leader performance in eight MegaCode scenarios. Videos of the scenarios were reviewed and rated by trained faculty facilitators and by a group of non-expert raters. The videos were reviewed “continuously” and “with pauses.” Two content experts served as the reference standard for grading, and four non-expert raters were used to test the reliability of the checklists.
Results
Our results demonstrate that non-expert raters are able to produce reliable grades when using the checklists under consideration, demonstrating excellent intra-rater reliability and agreement with a reference standard. The results also demonstrate that non-expert raters can be trained in the proper use of the checklist in a short amount of time, with no discernible learning curve thereafter. Finally, our results show that a single trained rater can achieve reliable scores of team leader performance during AHA MegaCodes when using our checklist in continuous mode, as measures of agreement in total scoring were very strong (Lin’s Concordance Correlation Coefficient = 0.96; Intraclass Correlation Coefficient = 0.97).
Discussion
We have shown that our checklists can yield reliable scores, are appropriate for use by non-expert raters, and are able to be employed during continuous assessment of team leader performance during the review of a simulated MegaCode. This checklist may be more appropriate for use by Advanced Cardiac Life Support (ACLS) instructors during MegaCode assessments than current tools provided by the AHA.
doi:10.1097/SIH.0b013e3182590b07
PMCID: PMC3467004  PMID: 22863996
simulation; education; checklist; reliability; ACLS
6.  Home Telehealth for Patients With Chronic Obstructive Pulmonary Disease (COPD) 
Executive Summary
In July 2010, the Medical Advisory Secretariat (MAS) began work on a Chronic Obstructive Pulmonary Disease (COPD) evidentiary framework, an evidence-based review of the literature surrounding treatment strategies for patients with COPD. This project emerged from a request by the Health System Strategy Division of the Ministry of Health and Long-Term Care that MAS provide them with an evidentiary platform on the effectiveness and cost-effectiveness of COPD interventions.
After an initial review of health technology assessments and systematic reviews of COPD literature, and consultation with experts, MAS identified the following topics for analysis: vaccinations (influenza and pneumococcal), smoking cessation, multidisciplinary care, pulmonary rehabilitation, long-term oxygen therapy, noninvasive positive pressure ventilation for acute and chronic respiratory failure, hospital-at-home for acute exacerbations of COPD, and telehealth (including telemonitoring and telephone support). Evidence-based analyses were prepared for each of these topics. For each technology, an economic analysis was also completed where appropriate. In addition, a review of the qualitative literature on patient, caregiver, and provider perspectives on living and dying with COPD was conducted, as were reviews of the qualitative literature on each of the technologies included in these analyses.
The Chronic Obstructive Pulmonary Disease Mega-Analysis series is made up of the following reports, which can be publicly accessed at the MAS website at: http://www.hqontario.ca/en/mas/mas_ohtas_mn.html.
Chronic Obstructive Pulmonary Disease (COPD) Evidentiary Framework
Influenza and Pneumococcal Vaccinations for Patients With Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Smoking Cessation for Patients With Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Community-Based Multidisciplinary Care for Patients With Stable Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Pulmonary Rehabilitation for Patients With Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Long-term Oxygen Therapy for Patients With Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Noninvasive Positive Pressure Ventilation for Acute Respiratory Failure Patients With Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Noninvasive Positive Pressure Ventilation for Chronic Respiratory Failure Patients With Stable Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Hospital-at-Home Programs for Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Home Telehealth for Patients With Chronic Obstructive Pulmonary Disease (COPD): An Evidence-Based Analysis
Cost-Effectiveness of Interventions for Chronic Obstructive Pulmonary Disease Using an Ontario Policy Model
Experiences of Living and Dying With COPD: A Systematic Review and Synthesis of the Qualitative Empirical Literature
For more information on the qualitative review, please contact Mita Giacomini at: http://fhs.mcmaster.ca/ceb/faculty_member_giacomini.htm.
For more information on the economic analysis, please visit the PATH website: http://www.path-hta.ca/About-Us/Contact-Us.aspx.
The Toronto Health Economics and Technology Assessment (THETA) collaborative has produced an associated report on patient preference for mechanical ventilation. For more information, please visit the THETA website: http://theta.utoronto.ca/static/contact.
Objective
The objective of this analysis was to conduct an evidence-based assessment of home telehealth technologies for patients with chronic obstructive pulmonary disease (COPD) in order to inform recommendations regarding the access and provision of these services in Ontario. This analysis was one of several analyses undertaken to evaluate interventions for COPD. The perspective of this assessment was that of the Ontario Ministry of Health and Long-Term Care, a provincial payer of medically necessary health care services.
Clinical Need: Condition and Target Population
Canada is facing an increase in chronic respiratory diseases due in part to its aging demographic. The projected increase in COPD will put a strain on health care payers and providers. There is therefore an increasing demand for telehealth services that improve access to health care services while maintaining or improving quality and equality of care. Many telehealth technologies however are in the early stages of development or diffusion and thus require study to define their application and potential harms or benefits. The Medical Advisory Secretariat (MAS) therefore sought to evaluate telehealth technologies for COPD.
Technology
Telemedicine (or telehealth) refers to using advanced information and communication technologies and electronic medical devices to support the delivery of clinical care, professional education, and health-related administrative services.
Generally there are 4 broad functions of home telehealth interventions for COPD:
to monitor vital signs or biological health data (e.g., oxygen saturation),
to monitor symptoms, medication, or other non-biologic endpoints (e.g., exercise adherence),
to provide information (education) and/or other support services (such as reminders to exercise or positive reinforcement), and
to establish a communication link between patient and provider.
These functions often require distinct technologies, although some devices can perform a number of these diverse functions. For the purposes of this review, MAS focused on home telemonitoring and telephone only support technologies.
Telemonitoring (or remote monitoring) refers to the use of medical devices to remotely collect a patient’s vital signs and/or other biologic health data and the transmission of those data to a monitoring station for interpretation by a health care provider.
Telephone only support refers to disease/disorder management support provided by a health care provider to a patient who is at home via telephone or videoconferencing technology in the absence of transmission of patient biologic data.
Research Questions
What is the effectiveness, cost-effectiveness, and safety of home telemonitoring compared with usual care for patients with COPD?
What is the effectiveness, cost-effectiveness, and safety of telephone only support programs compared with usual care for patients with COPD?
Research Methods
Literature Search
Search Strategy
A literature search was performed on November 3, 2010 using OVID MEDLINE, MEDLINE In-Process and Other Non-Indexed Citations, EMBASE, the Cumulative Index to Nursing & Allied Health Literature (CINAHL), the Cochrane Library, and the International Agency for Health Technology Assessment (INAHTA) for studies published from January 1, 2000 until November 3, 2010. Abstracts were reviewed by a single reviewer and, for those studies meeting the eligibility criteria, full-text articles were obtained. Reference lists were also examined for any additional relevant studies not identified through the search. Articles with unknown eligibility were reviewed with a second clinical epidemiologist, and then a group of epidemiologists until consensus was established. The quality of evidence was assessed as high, moderate, low, or very low according to GRADE methodology.
Inclusion Criteria – Question #1
frequent transmission of a patient’s physiological data collected at home and without a health care professional physically present to health care professionals for routine monitoring through the use of a communication technology;
monitoring combined with a coordinated management and feedback system based on transmitted data;
telemonitoring as a key component of the intervention (subjective determination);
usual care as provided by the usual care provider for the control group;
randomized controlled trials (RCTs), controlled clinical trials (CCTs), systematic reviews, and/or meta-analyses;
published between January 1, 2000 and November 3, 2010.
Inclusion Criteria – Question #2
scheduled or frequent contact between patient and a health care professional via telephone or videoconferencing technology in the absence of transmission of patient physiological data;
monitoring combined with a coordinated management and feedback system based on transmitted data;
telephone support as a key component of the intervention (subjective determination);
usual care as provided by the usual care provider for the control group;
RCTs, CCTs, systematic reviews, and/or meta-analyses;
published between January 1, 2000 and November 3, 2010.
Exclusion Criteria
published in a language other than English;
intervention group (and not control) receiving some form of home visits by a medical professional, typically a nurse (i.e., telenursing) beyond initial technology set-up and education, to collect physiological data, or to somehow manage or treat the patient;
not recording patient or health system outcomes (e.g., technical reports testing accuracy, reliability or other development-related outcomes of a device, acceptability/feasibility studies, etc.);
not using an independent control group that received usual care (e.g., studies employing historical or periodic controls).
Outcomes of Interest
hospitalizations (primary outcome)
mortality
emergency department visits
length of stay
quality of life
other […]
Subgroup Analyses (a priori)
length of intervention (primary)
severity of COPD (primary)
Quality of Evidence
The quality of evidence assigned to individual studies was determined using a modified CONSORT Statement Checklist for Randomized Controlled Trials. (1) The CONSORT Statement was adapted to include 3 additional quality measures: the adequacy of control group description, significant differential loss to follow-up between groups, and greater than or equal to 30% study attrition. Individual study quality was defined based on total scores according to the CONSORT Statement checklist: very low (0 to < 40%), low (≥ 40 to < 60%), moderate (≥ 60 to < 80%), and high (≥ 80 to 100%).
The quality of the body of evidence was assessed as high, moderate, low, or very low according to the GRADE Working Group criteria. The following definitions of quality were used in grading the quality of the evidence:
Summary of Findings
Six publications, representing 5 independent trials, met the eligibility criteria for Research Question #1. Three trials were RCTs reported across 4 publications, whereby patients were randomized to home telemonitoring or usual care, and 2 trials were CCTs, whereby patients or health care centers were nonrandomly assigned to intervention or usual care.
A total of 310 participants were studied across the 5 included trials. The mean age of study participants in the included trials ranged from 61.2 to 74.5 years for the intervention group and 61.1 to 74.5 years for the usual care group. The percentage of men ranged from 40% to 64% in the intervention group and 46% to 72% in the control group.
All 5 trials were performed in a moderate to severe COPD patient population. Three trials initiated the intervention following discharge from hospital. One trial initiated the intervention following a pulmonary rehabilitation program. The final trial initiated the intervention during management of patients at an outpatient clinic.
Four of the 5 trials included oxygen saturation (i.e., pulse oximetry) as one of the biological patient parameters being monitored. Additional parameters monitored included forced expiratory volume in one second, peak expiratory flow, and temperature.
There was considerable clinical heterogeneity between trials in study design, methods, and intervention/control. In relation to the telemonitoring intervention, 3 of the 5 included studies used an electronic health hub that performed multiple functions beyond the monitoring of biological parameters. One study used only a pulse oximeter device alone with modem capabilities. Finally, in 1 study, patients measured and then forwarded biological data to a nurse during a televideo consultation. Usual care varied considerably between studies.
Only one trial met the eligibility criteria for Research Question #2. The included trial was an RCT that randomized 60 patients to nurse telephone follow-up or usual care (no telephone follow-up). Participants were recruited from the medical department of an acute-care hospital in Hong Kong and began receiving follow-up after discharge from the hospital with a diagnosis of COPD (no severity restriction). The intervention itself consisted of only two 10-to 20-minute telephone calls, once between days 3 to 7 and once between days 14 to 20, involving a structured, individualized educational and supportive programme led by a nurse that focused on 3 components: assessment, management options, and evaluation.
Regarding Research Question #1:
Low to very low quality evidence (according to GRADE) finds non-significant effects or conflicting effects (of significant or non-significant benefit) for all outcomes examined when comparing home telemonitoring to usual care.
There is a trend towards significant increase in time free of hospitalization and use of other health care services with home telemonitoring, but these findings need to be confirmed further in randomized trials of high quality.
There is severe clinical heterogeneity between studies that limits summary conclusions.
The economic impact of home telemonitoring is uncertain and requires further study.
Home telemonitoring is largely dependent on local information technologies, infrastructure, and personnel, and thus the generalizability of external findings may be low. Jurisdictions wishing to replicate home telemonitoring interventions should likely test those interventions within their jurisdictional framework before adoption, or should focus on home-grown interventions that are subjected to appropriate evaluation and proven effective.
Regarding Research Question #2:
Low quality evidence finds significant benefit in favour of telephone-only support for self-efficacy and emergency department visits when compared to usual care, but non-significant results for hospitalizations and hospital length of stay.
There are very serious issues with the generalizability of the evidence and thus additional research is required.
PMCID: PMC3384362  PMID: 23074421
7.  Can the surgical checklist reduce the risk of wrong site surgery in orthopaedics? - can the checklist help? Supporting evidence from analysis of a national patient incident reporting system 
Background
Surgical procedures are now very common, with estimates ranging from 4% of the general population having an operation per annum in economically-developing countries; this rising to 8% in economically-developed countries. Whilst these surgical procedures typically result in considerable improvements to health outcomes, it is increasingly appreciated that surgery is a high risk industry. Tools developed in the aviation industry are beginning to be used to minimise the risk of errors in surgery. One such tool is the World Health Organization's (WHO) surgery checklist. The National Patient Safety Agency (NPSA) manages the largest database of patient safety incidents (PSIs) in the world, already having received over three million reports of episodes of care that could or did result in iatrogenic harm. The aim of this study was to estimate how many incidents of wrong site surgery in orthopaedics that have been reported to the NPSA could have been prevented by the WHO surgical checklist.
Methods
The National Reporting and Learning Service (NRLS) database was searched between 1st January 2008- 31st December 2008 to identify all incidents classified as wrong site surgery in orthopaedics. These incidents were broken down into the different types of wrong site surgery. A Likert-scale from 1-5 was used to assess the preventability of these cases if the checklist was used.
Results
133/316 (42%) incidents satisfied the inclusion criteria. A large proportion of cases, 183/316 were misclassified. Furthermore, there were fewer cases of actual harm [9% (12/133)] versus 'near-misses' [121/133 (91%)]. Subsequent analysis revealed a smaller proportion of 'near-misses' being prevented by the checklist than the proportion of incidents that resulted in actual harm; 18/121 [14.9% (95% CI 8.5 - 21.2%)] versus 10/12 [83.3% (95%CI 62.2 - 104.4%)] respectively. Summatively, the checklist could have been prevented 28/133 [21.1% (95%CI 14.1 - 28.0%)] patient safety incidents.
Discussion
Orthopaedic surgery is a high volume specialty with major technical complexity in terms of equipment demands and staff training and familiarity. There is therefore an increased propensity for errors to occur. Wrong-site surgery still occurs in this specialty and is a potentially devastating situation for both the patient and surgeon. Despite the limitations of inclusion and reporting bias, our study highlights the need to match technical precision with patient safety. Tools such as the WHO surgical checklist can help us to achieve this.
doi:10.1186/1749-799X-6-18
PMCID: PMC3101645  PMID: 21501466
8.  Interactions between Non-Physician Clinicians and Industry: A Systematic Review 
PLoS Medicine  2013;10(11):e1001561.
In a systematic review of studies of interactions between non-physician clinicians and industry, Quinn Grundy and colleagues found that many of the issues identified for physicians' industry interactions exist for non-physician clinicians.
Please see later in the article for the Editors' Summary
Background
With increasing restrictions placed on physician–industry interactions, industry marketing may target other health professionals. Recent health policy developments confer even greater importance on the decision making of non-physician clinicians. The purpose of this systematic review is to examine the types and implications of non-physician clinician–industry interactions in clinical practice.
Methods and Findings
We searched MEDLINE and Web of Science from January 1, 1946, through June 24, 2013, according to PRISMA guidelines. Non-physician clinicians eligible for inclusion were: Registered Nurses, nurse prescribers, Physician Assistants, pharmacists, dieticians, and physical or occupational therapists; trainee samples were excluded. Fifteen studies met inclusion criteria. Data were synthesized qualitatively into eight outcome domains: nature and frequency of industry interactions; attitudes toward industry; perceived ethical acceptability of interactions; perceived marketing influence; perceived reliability of industry information; preparation for industry interactions; reactions to industry relations policy; and management of industry interactions. Non-physician clinicians reported interacting with the pharmaceutical and infant formula industries. Clinicians across disciplines met with pharmaceutical representatives regularly and relied on them for practice information. Clinicians frequently received industry “information,” attended sponsored “education,” and acted as distributors for similar materials targeted at patients. Clinicians generally regarded this as an ethical use of industry resources, and felt they could detect “promotion” while benefiting from industry “information.” Free samples were among the most approved and common ways that clinicians interacted with industry. Included studies were observational and of varying methodological rigor; thus, these findings may not be generalizable. This review is, however, the first to our knowledge to provide a descriptive analysis of this literature.
Conclusions
Non-physician clinicians' generally positive attitudes toward industry interactions, despite their recognition of issues related to bias, suggest that industry interactions are normalized in clinical practice across non-physician disciplines. Industry relations policy should address all disciplines and be implemented consistently in order to mitigate conflicts of interest and address such interactions' potential to affect patient care.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Making and selling health care goods (including drugs and devices) and services is big business. To maximize the profits they make for their shareholders, companies involved in health care build relationships with physicians by providing information on new drugs, organizing educational meetings, providing samples of their products, giving gifts, and holding sponsored events. These relationships help to keep physicians informed about new developments in health care but also create the potential for causing harm to patients and health care systems. These relationships may, for example, result in increased prescription rates of new, heavily marketed medications, which are often more expensive than their generic counterparts (similar unbranded drugs) and that are more likely to be recalled for safety reasons than long-established drugs. They may also affect the provision of health care services. Industry is providing an increasingly large proportion of routine health care services in many countries, so relationships built up with physicians have the potential to influence the commissioning of the services that are central to the treatment and well-being of patients.
Why Was This Study Done?
As a result of concerns about the tension between industry's need to make profits and the ethics underlying professional practice, restrictions are increasingly being placed on physician–industry interactions. In the US, for example, the Physician Payments Sunshine Act now requires US manufacturers of drugs, devices, and medical supplies that participate in federal health care programs to disclose all payments and gifts made to physicians and teaching hospitals. However, other health professionals, including those with authority to prescribe drugs such as pharmacists, Physician Assistants, and nurse practitioners are not covered by this legislation or by similar legislation in other settings, even though the restructuring of health care to prioritize primary care and multidisciplinary care models means that “non-physician clinicians” are becoming more numerous and more involved in decision-making and medication management. In this systematic review (a study that uses predefined criteria to identify all the research on a given topic), the researchers examine the nature and implications of the interactions between non-physician clinicians and industry.
What Did the Researchers Do and Find?
The researchers identified 15 published studies that examined interactions between non-physician clinicians (Registered Nurses, nurse prescribers, midwives, pharmacists, Physician Assistants, and dieticians) and industry (corporations that produce health care goods and services). They extracted the data from 16 publications (representing 15 different studies) and synthesized them qualitatively (combined the data and reached word-based, rather than numerical, conclusions) into eight outcome domains, including the nature and frequency of interactions, non-physician clinicians' attitudes toward industry, and the perceived ethical acceptability of interactions. In the research the authors identified, non-physician clinicians reported frequent interactions with the pharmaceutical and infant formula industries. Most non-physician clinicians met industry representatives regularly, received gifts and samples, and attended educational events or received educational materials (some of which they distributed to patients). In these studies, non-physician clinicians generally regarded these interactions positively and felt they were an ethical and appropriate use of industry resources. Only a minority of non-physician clinicians felt that marketing influenced their own practice, although a larger percentage felt that their colleagues would be influenced. A sizeable proportion of non-physician clinicians questioned the reliability of industry information, but most were confident that they could detect biased information and therefore rated this information as reliable, valuable, or useful.
What Do These Findings Mean?
These and other findings suggest that non-physician clinicians generally have positive attitudes toward industry interactions but recognize issues related to bias and conflict of interest. Because these findings are based on a small number of studies, most of which were undertaken in the US, they may not be generalizable to other countries. Moreover, they provide no quantitative assessment of the interaction between non-physician clinicians and industry and no information about whether industry interactions affect patient care outcomes. Nevertheless, these findings suggest that industry interactions are normalized (seen as standard) in clinical practice across non-physician disciplines. This normalization creates the potential for serious risks to patients and health care systems. The researchers suggest that it may be unrealistic to expect that non-physician clinicians can be taught individually how to interact with industry ethically or how to detect and avert bias, particularly given the ubiquitous nature of marketing and promotional materials. Instead, they suggest, the environment in which non-physician clinicians practice should be structured to mitigate the potentially harmful effects of interactions with industry.
Additional Information
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001561.
This study is further discussed in a PLOS Medicine Perspective by James S. Yeh and Aaron S. Kesselheim
The American Medical Association provides guidance for physicians on interactions with pharmaceutical industry representatives, information about the Physician Payments Sunshine Act, and a toolkit for preparing Physician Payments Sunshine Act reports
The International Council of Nurses provides some guidance on industry interactions in its position statement on nurse-industry relations
The UK General Medical Council provides guidance on financial and commercial arrangements and conflicts of interest as part of its good medical practice website, which describes what is required of all registered doctors in the UK
Understanding and Responding to Pharmaceutical Promotion: A Practical Guide is a manual prepared by Health Action International and the World Health Organization that schools of medicine and pharmacy can use to train students how to recognize and respond to pharmaceutical promotion.
The Institute of Medicine's Report on Conflict of Interest in Medical Research, Education, and Practice recommends steps to identify, limit, and manage conflicts of interest
The University of California, San Francisco, Office of Continuing Medical Education offers a course called Marketing of Medicines
doi:10.1371/journal.pmed.1001561
PMCID: PMC3841103  PMID: 24302892
9.  Surgical checklists: the human factor 
Background
Surgical checklists has been shown to improve patient safety and teamwork in the operating theatre. However, despite the known benefits of the use of checklists in surgery, in some cases the practical implementation has been found to be less than universal. A questionnaire methodology was used to quantitatively evaluate the attitudes of theatre staff towards a modified version of the World Health Organisation (WHO) surgical checklist with relation to: beliefs about levels of compliance and support, impact on patient safety and teamwork, and barriers to the use of the checklist.
Methods
Using the theory of planned behaviour as a framework, 14 semi-structured interviews were conducted with theatre personnel regarding their attitudes towards, and levels of compliance with, a checklist. Based upon the interviews, a 27-item questionnaire was developed and distribute to all theatre personnel in an Irish hospital.
Results
Responses were obtained from 107 theatre staff (42.6% response rate). Particularly for nurses, the overall attitudes towards the effect of the checklist on safety and teamworking were positive. However, there was a lack of rigour with which the checklist was being applied. Nurses were significantly more sensitive to the barriers to the use of the checklist than anaesthetists or surgeons. Moreover, anaesthetists were not as positively disposed to the surgical checklist as surgeons and nurse. This finding was attributed to the tendency for the checklist to be completed during a period of high workload for the anaesthetists, resulting in a lack of engagement with the process.
Conclusion
In order to improve the rigour with which the surgical checklist is applied, there is a need for: the involvement of all members of the theatre team in the checklist process, demonstrated support for the checklist from senior personnel, on-going education and training, and barriers to the implementation of the checklist to be addressed.
doi:10.1186/1754-9493-7-14
PMCID: PMC3669630  PMID: 23672665
Surgical checklist; Surgery; Patient safety
10.  Becoming a Good Doctor: Perceived Need for Ethics Training Focused on Practical and Professional Development Topics 
Objective
Ethics training has become a core component of medical student and resident education. Curricula have been developed without the benefit of data regarding the views of physicians-in-training on the need for ethics instruction that focuses on practical issues and professional development topics.
Methods
A written survey was sent to all medical students and PGY1-3 residents at the University of New Mexico School of Medicine. The survey consisted of eight demographic questions and 124 content questions in 10 domains. Responses to a set of 24 items related to ethically important dilemmas, which may occur in the training period and subsequent professional practice, are reported. Items were each rated on a 9-point scale addressing the level of educational attention needed compared to the amount currently provided.
Results
Survey respondents included 200 medical students (65% response) and 136 residents (58% response). Trainees, regardless of level of training or clinical discipline, perceived a need for more academic attention directed at practical ethical and professional dilemmas present during training and the practice of medicine. Women expressed a desire for more education directed at both training-based and practice-based ethical dilemmas when compared to men. A simple progression of interest in ethics topics related to level of medical training was not found. Residents in diverse clinical specialties differed in, perceived ethics educational needs. Psychiatry residents reported a need for enhanced education directed toward training stage ethics problems.
Conclusions
This study documents the importance placed on ethics education directed at practical real-world dilemmas and ethically important professional developmental issues by physicians-in-training. Academic medicine may be better able to fulfill its responsibilities in teaching ethics and professionalism and in serving its trainees by paying greater attention to these topics in undergraduate and graduate medical curricula.
doi:10.1176/appi.ap.29.3.301
PMCID: PMC1599855  PMID: 16141129
11.  The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study 
Quality of Life Research  2010;19(4):539-549.
Background
Aim of the COSMIN study (COnsensus-based Standards for the selection of health status Measurement INstruments) was to develop a consensus-based checklist to evaluate the methodological quality of studies on measurement properties. We present the COSMIN checklist and the agreement of the panel on the items of the checklist.
Methods
A four-round Delphi study was performed with international experts (psychologists, epidemiologists, statisticians and clinicians). Of the 91 invited experts, 57 agreed to participate (63%). Panel members were asked to rate their (dis)agreement with each proposal on a five-point scale. Consensus was considered to be reached when at least 67% of the panel members indicated ‘agree’ or ‘strongly agree’.
Results
Consensus was reached on the inclusion of the following measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing and cross-cultural validity), criterion validity, responsiveness, and interpretability. The latter was not considered a measurement property. The panel also reached consensus on how these properties should be assessed.
Conclusions
The resulting COSMIN checklist could be useful when selecting a measurement instrument, peer-reviewing a manuscript, designing or reporting a study on measurement properties, or for educational purposes.
doi:10.1007/s11136-010-9606-8
PMCID: PMC2852520  PMID: 20169472
Delphi technique; Outcome assessment; Psychometrics; Quality of life; Questionnaire
12.  Patient Safety Knowledge and Its Determinants in Medical Trainees 
Journal of General Internal Medicine  2007;22(8):1150-1154.
Background
Patient safety is a core educational topic for medical trainees.
Objectives
To determine the current level and determinants of patient safety knowledge in medical trainees.
Design
Multi-institutional cross-sectional assessment of patient safety knowledge.
Participants
Residents and medical students from seven Harvard-affiliated residencies and two Harvard Medical School courses.
Measurements
Participants were administered a 14-item validated test instrument developed based on the patient safety curriculum of the Risk Management Foundation (Cambridge, MA). The primary outcome measure was the amount of patient safety knowledge demonstrated by trainees on the validated test instrument. The secondary outcome measure was their subjective perceptions as to their baseline knowledge level in this domain.
Results
Ninety-two percent (640/693) of residents and medical students completed the patient safety test. Participants correctly answered a mean 58.4% of test items (SD 15.5%). Univariate analyses show that patient safety knowledge levels varied significantly by year of training (p = 0.001), degree program (p < 0.001), specialty (p < 0.001), country of medical school (p = 0.006), age (p < 0.001), and gender (p = 0.050); all but the latter two determinants remained statistically significant in multivariate models. In addition, trainees were unable to assess their own knowledge deficiencies in this domain.
Conclusions
Patient safety knowledge is limited among medical trainees across a broad range of training levels, degrees, and specialties. Effective educational interventions that target deficiencies in patient safety knowledge are greatly needed.
doi:10.1007/s11606-007-0247-8
PMCID: PMC2305739  PMID: 17551796
safety; medical errors; medical education
13.  Validation of core competencies during residency training in anaesthesiology 
Background and goal: Curriculum development for residency training is increasingly challenging in times of financial restrictions and time limitations. Several countries have adopted the CanMEDS framework for medical education as a model into their curricula of specialty training. The purpose of the present study was to validate the competency goals, as derived from CanMEDS, of the Department of Anaesthesiology and Intensive Care Medicine of the Berlin Charité University Medical Centre, by conducting a staff survey. These goals for the qualification of specialists stipulate demonstrable competencies in seven areas: expert medical action, efficient collaboration in a team, communications with patients and family, management and organisation, lifelong learning, professional behaviour, and advocacy of good health. We had previously developed a catalogue of curriculum items based on these seven core competencies. In order to evaluate the validity of this catalogue, we surveyed anaesthetists at our department in regard to their perception of the importance of each of these items. In addition to the descriptive acquisition of data, it was intended to assess the results of the survey to ascertain whether there were differences in the evaluation of these objectives by specialists and registrars.
Methods: The questionnaire with the seven adapted CanMEDS Roles included items describing each of their underlying competencies. Each anaesthetist (registrars and specialists) working at our institution in May of 2007 was asked to participate in the survey. Individual perception of relevance was rated for each item on a scale similar to the Likert system, ranging from 1 (highly relevant) to 5 (not at all relevant), from which ratings means were calculated. For determination of reliability, we calculated Cronbach’s alpha. To assess differences between subgroups, we performed analysis of variance.
Results: All seven roles were rated as relevant. Three of the seven competency goals (expert medical action, efficient collaboration in a team, and communication with patients and family) achieved especially high ratings. Only a few items differed significantly in their average rating between specialists and registrars.
Conclusions: We succeeded in validating the relevance of the adapted seven CanMEDS competencies for residency training within our institution. So far, many countries have adopted the Canadian Model, which indicates the great practicability of this competency-based model in curriculum planning. Roles with higher acceptance should be prioritised in existing curricula. It would be desirable to develop and validate a competency-based curriculum for specialty training in anaesthesiology throughout Germany by conducting a national survey to include specialists as well as registrars in curriculum development.
doi:10.3205/000146
PMCID: PMC3172723  PMID: 21921997
clinical competence; physicians; medical education; questionnaires; attitude of health personnel; curriculum
14.  Validation of an Instrument to Assess Evidence-Based Practice (EBP) Knowledge, Attitudes, Access and Confidence 
Journal of dental education  2011;75(2):131-144.
Purpose
This paper reports the validation of an assessment instrument designed to measure the outcomes of training in evidence-based practice (EBP) within the context of dentistry. Four EBP dimensions are measured by this instrument: (1) understanding of EBP concepts, (2) attitudes about EBP, (3) evidence accessing methods, and, (4) confidence in critical appraisal. The instrument is the Evidence Based Practice Knowledge, Attitudes, Access, and Confidence Evaluation (KACE) that has four scales, totaling 35 items: EBP knowledge (10), EBP attitudes (10), accessing evidence (9) and confidence in critical appraisal (6).
Methods
Four elements of validity were assessed: consistency of items within the KACE scales (extent to which items within a scale measure the same dimension), discrimination (capacity to detect differences between individuals with different training or experience), responsiveness (capacity to detect the effects of education on trainees) and test-retest reliability. Internal consistency of scales was assessed by analyzing responses from KACEs completed by second year dental students, dental residents and dental school faculty using Cronbach alpha. Discriminative validity was assessed by comparing KACE scores for students, residents and faculty members. Responsiveness was assessed by comparing pre - and post - training responses for dental students and residents. To measure test-retest reliability, the KACE was completed twice by a class of freshmen dental students 17 days apart and the knowledge scale was completed twice by 16 dental faculty 14 days apart.
Results
Item - to - scale consistency ranged from 0.21 to 0.78 for knowledge, 0.57 to 0.83 for attitude, 0.70 to 0.84 for accessing evidence and 0.87 to 0.94 for confidence. For discrimination, ANOVA and post-hoc testing by the Tukey-Kramer method revealed significant score differences among students, residents and faculty consistent with education and experience levels. For responsiveness to training, dental students and residents demonstrated statistically significant changes, in desired directions, from pre - to post - test. For the student test-retest, Pearson correlations for KACE scales were: knowledge (0.66), attitudes (0.66), evidence accessing (0.74) and confidence (0.76). For the knowledge scale test-retest by faculty, the Pearson correlation was 0.79.
Conclusion
The construct validity of the KACE is equivalent to that of instruments that assess similar EBP dimensions in medicine. Item consistency for the knowledge scale was more variable than for other KACE scales, a finding also reported for medically-oriented EBP instruments. The KACE has good discriminative validity, responsiveness to training effects, and test-retest reliability.
PMCID: PMC3078051  PMID: 21293036
Evidence-based practice; critical appraisal; dental education; assessment
15.  Microsurgical lab testing is a reliable method for assessing ophthalmology residents' surgical skills 
The British Journal of Ophthalmology  2007;91(12):1691-1694.
Background
Formal assessment of clinical competencies is necessary to ensure that all residents are acquiring important skills and, in the United States, will soon become a requirement for residency programme accreditation by the Accreditation Council for Graduate Medical Education (ACGME). The Eye Surgical Skills Assessment Test (ESSAT), a laboratory‐based surgical skills obstacle course, was developed in response to the need for improved tools for the assessment of surgical skills during residency. The ESSAT has previously been shown to have face and content validity, and in this study we sought to determine its inter‐rater reliability and, to some extent, its construct validity.
Methods
Twenty‐seven content experts (residency programme directors and faculty members involved with resident surgical training) watched videos of a junior resident and senior resident completing the three ESSAT stations (skin suturing, muscle recession, and phacoemulsification: wound construction & suturing technique) and completed assessment forms, both task‐specific checklists and a global rating scale of performance.
Results
The ESSAT showed strong inter‐rater reliability for determining whether a resident “passed” a threshold of competency at each station for both the checklists and global rating scale. In addition, for each station, the senior resident was consistently rated above a “passing” threshold using either assessment form, whereas the junior resident was more often rated below (94% vs 30% passing on completed forms).
Conclusion
These results, along with the findings of our face and content validity analysis, support the reliability and validity of the ESSAT, and indicate that it could be a useful tool for improving the assessment of surgical skill during residency. The ESSAT is a tool that all residency programmes could implement as a part of their ophthalmic surgical curriculum and competency assessment, and may be useful to set a threshold of competence that all residents would need to achieve prior to entering the operating room.
doi:10.1136/bjo.2007.123083
PMCID: PMC2095537  PMID: 17591670
16.  Implementation of checklists in health care; learning from high-reliability organisations 
Background
Checklists are common in some medical fields, including surgery, intensive care and emergency medicine. They can be an effective tool to improve care processes and reduce mortality and morbidity. Despite the seemingly rapid acceptance and dissemination of the checklist, there are few studies describing the actual process of developing and implementing such tools in health care. The aim of this study is to explore the experiences from checklist development and implementation in a group of non-medical, high reliability organisations (HROs).
Method
A qualitative study based on key informant interviews and field visits followed by a Delphi approach. Eight informants, each with 10-30 years of checklist experience, were recruited from six different HROs.
Results
The interviews generated 84 assertions and recommendations for checklist implementation. To achieve checklist acceptance and compliance, there must be a predefined need for which a checklist is considered a well suited solution. The end-users ("sharp-end") are the key stakeholders throughout the development and implementation process. Proximity and ownership must be assured through a thorough and wise process. All informants underlined the importance of short, self-developed, and operationally-suited checklists. Simulation is a valuable and widely used method for training, revision, and validation.
Conclusion
Checklists have been a cornerstone of safety management in HROs for nearly a century, and are becoming increasingly popular in medicine. Acceptance and compliance are crucial for checklist implementation in health care. Experiences from HROs may provide valuable input to checklist implementation in healthcare.
doi:10.1186/1757-7241-19-53
PMCID: PMC3205016  PMID: 21967747
17.  Systematic review of safety checklists for use by medical care teams in acute hospital settings - limited evidence of effectiveness 
Background
Patient safety is a fundamental component of good quality health care. Checklists have been proposed as a method of improving patient safety. This systematic review, asked "In acute hospital settings, would the use of safety checklists applied by medical care teams, compared to not using checklists, improve patient safety?"
Methods
We searched the Cochrane Library, MEDLINE, CINAHL, and EMBASE for randomised controlled trials published in English before September 2009. Studies were selected and appraised by two reviewers independently in consultation with colleagues, using inclusion, exclusion and appraisal criteria established a priori.
Results
Nine cohort studies with historical controls studies from four hospital care settings were included-intensive care unit, emergency department, surgery, and acute care. The studies used a variety of designs of safety checklists, and implemented them in different ways, however most incorporated an educational component to teach the staff how to use the checklist. The studies assessed outcomes occurring a few weeks to a maximum of 12 months post-implementation, and these outcomes were diverse.
The studies were generally of low to moderate quality and of low levels of evidence, with all but one of the studies containing a high risk of bias.
The results of these studies suggest some improvements in patient safety arising from use of safety checklists, but these were not consistent across all studies or for all outcomes. Some studies showed no difference in outcomes between checklist use and standard care without a checklist. Due to the variations in setting, checklist design, educational training given, and outcomes measured, it was unfeasible to accurately summarise any trends across all studies.
Conclusions
The included studies suggest some benefits of using safety checklists to improve protocol adherence and patient safety, but due to the risk of bias in these studies, their results should be interpreted with caution. More high quality and studies, are needed to enable confident conclusions about the effectiveness of safety checklists in acute hospital settings.
doi:10.1186/1472-6963-11-211
PMCID: PMC3176176  PMID: 21884618
18.  Effect of an Office-Based Surgical Safety System on Patient Outcomes 
Eplasty  2012;12:e59.
Objective: To implement a customizable checklist in an interdisciplinary, team-based plastic surgery setting to reduce surgical complications. Methods: We examined the effects on patient outcomes and documentation of a customizable, office-based surgical safety checklist. On the basis of the World Health Organization Surgical Safety Checklist, we developed a 28-element, perioperative checklist for use in the office-based surgical setting. The checklist was implemented in an office-based plastic surgery practice with an already high standard of care. We recorded baseline, prechecklist rates for each checklist item and postoperative adverse outcomes via a retrospective chart review of 219 cases. After an education program and 30-day run-in period, a prospective, post–checklist implementation chart review was initiated (n = 184), with outcome data compared to the baseline. Results: The total number of complications per 100 patients decreased from 15.1 to 2.72 after checklist implementation (P < .0001), for an absolute risk reduction of 12.4. The proportion of patients with one or more complications decreased from 11.9% to 2.72% (P = .0006). Site and side marking increased from 69.9% prechecklist to 97.8% (P < .0001). Medical optimization increased from 90.9% to 99.5% (P < .0001). Emergency medical services (EMS) policy confirmation, case-specific equipment availability, anticipation of estimated blood loss, and verbal confirmation of local anesthetic toxicity precautions increased from 0% to 90.0% (P < .0001), 92.4% (P < .0001), 82.1% (P < .0001), and 91.3% (P < .0001), respectively. Assessment of patient satisfaction increased from 57.1% to 90.8% (P < .0001). Conclusions: Implementation of a customizable checklist was associated with a reduction in surgical complications in an office-based plastic surgery practice with an already high standard of care.
PMCID: PMC3536439  PMID: 23308306
19.  Changes in postgraduate medical education and training in clinical radiology 
Postgraduate medical education and training in many specialties, including Clinical Radiology, is undergoing major changes. In part this is to ensure that shorter training periods maximise the learning opportunities but it is also to bring medical education in line with broader educational theory. Learning outcomes need to be defined so that there is no doubt what knowledge, skills, attitudes and behaviours are expected of those in training. Curricula should be developed into competency or outcome based models and should state the aims, objectives, content, outcomes and processes of a training programme. They should include a description of the methods of learning, teaching, feedback and supervision. Assessment systems must be matched to the curriculum and must be fair, reliable and valid. Workplace based assessments including the use of multisource feedback need to be developed and validated for use during radiology training. These should be used in a formative and developmental way, although the overall results from a series of such assessments can be used in a more summative way to determine progress to the next phase of training. Formal standard setting processes need to be established for ‘high stakes’ summative assessments such as examinations. In addition the unique skills required of a radiologist in terms of image interpretation, pattern recognition, deduction and diagnosis need to be evaluated in robust, reliable and valid ways. Through a combination of these methods we can be assured that decisions about trainees’ progression through training is fair and standardised and that we are protecting patients by establishing national standards for training, curricula and assessment methods.
doi:10.2349/biij.4.1.e19
PMCID: PMC3097704  PMID: 21614310
Postgraduate; radiology; training; education
20.  Reporting Guidelines for Survey Research: An Analysis of Published Guidance and Reporting Practices 
PLoS Medicine  2011;8(8):e1001069.
Carol Bennett and colleagues review the evidence and find that there is limited guidance and no consensus on the optimal reporting of survey research.
Background
Research needs to be reported transparently so readers can critically assess the strengths and weaknesses of the design, conduct, and analysis of studies. Reporting guidelines have been developed to inform reporting for a variety of study designs. The objective of this study was to identify whether there is a need to develop a reporting guideline for survey research.
Methods and Findings
We conducted a three-part project: (1) a systematic review of the literature (including “Instructions to Authors” from the top five journals of 33 medical specialties and top 15 general and internal medicine journals) to identify guidance for reporting survey research; (2) a systematic review of evidence on the quality of reporting of surveys; and (3) a review of reporting of key quality criteria for survey research in 117 recently published reports of self-administered surveys. Fewer than 7% of medical journals (n = 165) provided guidance to authors on survey research despite a majority having published survey-based studies in recent years. We identified four published checklists for conducting or reporting survey research, none of which were validated. We identified eight previous reviews of survey reporting quality, which focused on issues of non-response and accessibility of questionnaires. Our own review of 117 published survey studies revealed that many items were poorly reported: few studies provided the survey or core questions (35%), reported the validity or reliability of the instrument (19%), defined the response rate (25%), discussed the representativeness of the sample (11%), or identified how missing data were handled (11%).
Conclusions
There is limited guidance and no consensus regarding the optimal reporting of survey research. The majority of key reporting criteria are poorly reported in peer-reviewed survey research articles. Our findings highlight the need for clear and consistent reporting guidelines specific to survey research.
Please see later in the article for the Editors' Summary
Editors' Summary
Background
Surveys, or questionnaires, are an essential component of many types of research, including health, and usually gather information by asking a sample of people questions on a specific topic and then generalizing the results to a larger population. Surveys are especially important when addressing topics that are difficult to assess using other approaches and usually rely on self reporting, for example self-reported behaviors, such as eating habits, satisfaction, beliefs, knowledge, attitudes, opinions. However, the methods used in conducting survey research can significantly affect the reliability, validity, and generalizability of study results, and without clear reporting of the methods used in surveys, it is difficult or impossible to assess these characteristics and therefore to have confidence in the findings.
Why Was This Study Done?
This uncertainty in other forms of research has given rise to Reporting Guidelines—evidence-based, validated tools that aim to improve the reporting quality of health research. The STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) Statement includes cross-sectional studies, which often involve surveys. But not all surveys are epidemiological, and STROBE does not include methods' and results' reporting characteristics that are unique to surveys. Therefore, the researchers conducted this study to help determine whether there is a need for a reporting guideline for health survey research.
What Did the Researchers Do and Find?
The researchers identified any previous relevant guidance for survey research, and any evidence on the quality of reporting of survey research, by: reviewing current guidance for reporting survey research in the “Instructions to Authors” of leading medical journals and in published literature; conducting a systematic review of evidence on the quality of reporting of surveys; identifying key quality criteria for the conduct of survey research; and finally, reviewing how these criteria are currently reported by conducting a review of recently published reports of self-administered surveys.
The researchers found that 154 of the 165 journals searched (93.3%) did not provide any guidance on survey reporting, even though the majority (81.8%) have published survey research. Only three of the 11 journals that provided some guidance gave more than one directive or statement. Five papers and one Internet site provided guidance on the reporting of survey research, but none used validated measures or explicit methods for development. The researchers identified eight papers that addressed the quality of reporting of some aspect of survey research: the reporting of response rates; the reporting of non-response analyses in survey research; and the degree to which authors make their survey instrument available to readers. In their review of 117 published survey studies, the researchers found that many items were poorly reported: few studies provided the survey or core questions (35%), reported the validity or reliability of the instrument (19%), discussed the representativeness of the sample (11%), or identified how missing data were handled (11%). Furthermore, (88 [75%]) did not include any information on consent procedures for research participants, and one-third (40 [34%]) of papers did not report whether the study had received research ethics board review.
What Do These Findings Mean?
Overall, these results show that guidance is limited and consensus lacking about the optimal reporting of survey research, and they highlight the need for a well-developed reporting guideline specifically for survey research—possibly an extension of the guideline for observational studies in epidemiology (STROBE)—that will provide the structure to ensure more complete reporting and allow clearer review and interpretation of the results from surveys.
Additional Information
Please access these web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001069.
More than 100 reporting guidelines covering a broad spectrum of research types are indexed on the EQUATOR Networks web site
More information about STROBE is available on the STROBE Statement web site
doi:10.1371/journal.pmed.1001069
PMCID: PMC3149080  PMID: 21829330
21.  Use of a Standardized Patient Exercise to Assess Core Competencies During Fellowship Training 
Background
The Accreditation Council for Graduate Medical Education requires fellows in many specialties to demonstrate attainment of 6 core competencies, yet relatively few validated assessment tools currently exist. We present our initial experience with the design and implementation of a standardized patient (SP) exercise during gastroenterology fellowship that facilitates appraisal of all core clinical competencies.
Methods
Fellows evaluated an SP trained to portray an individual referred for evaluation of abnormal liver tests. The encounters were independently graded by the SP and a faculty preceptor for patient care, professionalism, and interpersonal and communication skills using quantitative checklist tools. Trainees' consultation notes were scored using predefined key elements (medical knowledge) and subjected to a coding audit (systems-based practice). Practice-based learning and improvement was addressed via verbal feedback from the SP and self-assessment of the videotaped encounter.
Results
Six trainees completed the exercise. Second-year fellows received significantly higher scores in medical knowledge (55.0 ± 4.2 [standard deviation], P  =  .05) and patient care skills (19.5 ± 0.7, P  =  .04) by a faculty evaluator as compared with first-year trainees (46.2 ± 2.3 and 14.7 ± 1.5, respectively). Scores correlated by Spearman rank (0.82, P  =  .03) with the results of the Gastroenterology Training Examination. Ratings of the fellows by the SP did not differ by level of training, nor did they correlate with faculty scores. Fellows viewed the exercise favorably, with most indicating they would alter their practice based on the experience.
Conclusions
An SP exercise is an efficient and effective tool for assessing core clinical competencies during fellowship training.
doi:10.4300/JGME-D-09-00001.1
PMCID: PMC2931209  PMID: 21975896
22.  A comparison of the sports safety policies and practices of community sports clubs during training and competition in northern Sydney, Australia 
Objectives: To compare the safety policies and practices reported to be adopted during training and competition by community sports clubs in northern Sydney, Australia.
Methods: This cross sectional study involved face to face interviews, using an 81 item extensively validated questionnaire, with representatives of 163 community netball, rugby league, rugby union, and soccer clubs (response rate 85%). The study was undertaken during the winter sports season of 2000. Two separate 14 item scales were developed to analyse the level of safety policy adoption and safety practice implementation during training and competition. The statistical analysis comprised descriptive and inferential analysis stratified by sport.
Results: The reliability of the scales was good: Cronbach's α = 0.70 (competition scale) to 0.81 (training scale). Significant differences were found between the safety scores for training and competition for all clubs (mean difference 11.2; 95% confidence interval (CI) 10.0 to 12.5) and for each of the four sports: netball (mean difference 14.9; 95% CI 12.6 to 17.2); rugby league (mean difference 10.3; 95% CI 7.1 to 13.6); rugby union (mean difference 9.4; 95% CI 7.1 to 11.7); and soccer (mean difference 8.4; 95% CI 6.5 to 10.3).
Conclusions: The differences in the mean competition and training safety scores were significant for all sports. This indicates that safety policies were less often adopted and practices less often implemented during training than during competition. As injuries do occur at training, and sports participants often spend considerably more time training than competing, sporting bodies should consider whether the safety policies and practices adopted and implemented at training are adequate.
doi:10.1136/bjsm.2002.003806
PMCID: PMC1724743  PMID: 14751948
23.  How equity is addressed in clinical practice guidelines: a content analysis 
BMJ Open  2014;4(12):e005660.
Objectives
Considering equity into guidelines presents methodological challenges. This study aims to qualitatively synthesise the methods for incorporating equity in clinical practice guidelines (CPGs).
Setting
Content analysis of methodological publications.
Eligibility criteria for selecting studies
Methodological publications were included if they provided checklists/frameworks on when, how and to what extent equity should be incorporated in CPGs.
Data sources
We electronically searched MEDLINE, retrieved references, and browsed guideline development organisation websites from inception to January 2013. After study selection by two authors, general characteristics and checklists items/framework components from included studies were extracted. Based on the questions or items from checklists/frameworks (unit of analysis), content analysis was conducted to identify themes and questions/items were grouped into these themes.
Primary outcomes
The primary outcomes were methodological themes and processes on how to address equity issues in guideline development.
Results
8 studies with 10 publications were included from 3405 citations. In total, a list of 87 questions/items was generated from 17 checklists/frameworks. After content analysis, questions were grouped into eight themes (‘scoping questions’, ‘searching relevant evidence’, ‘appraising evidence and recommendations’, ‘formulating recommendations’, ‘monitoring implementation’, ‘providing a flow chart to include equity in CPGs’, and ‘others: reporting of guidelines and comments from stakeholders’ for CPG developers and ‘assessing the quality of CPGs’ for CPG users). Four included studies covered more than five of these themes. We also summarised the process of guideline development based on the themes mentioned above.
Conclusions
For disadvantaged population-specific CPGs, eight important methodological issues identified in this review should be considered when including equity in CPGs under the guidance of a scientific guideline development manual.
doi:10.1136/bmjopen-2014-005660
PMCID: PMC4265087  PMID: 25479795
QUALITATIVE RESEARCH; PUBLIC HEALTH
24.  Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitals - a validation study 
Background
Improving patient safety has become a major focus of clinical care and research over the past two decades. An institution’s patient safety climate represents an essential component of ensuring a safe environment and thereby can be vital to the prevention of adverse events. Covering six patient safety related factors, the Safety Attitudes Questionnaire (SAQ) is a validated and widely used instrument to measure the patient safety climate in clinical areas. The objective of this study was to assess the psychometric properties of the German language version of the SAQ.
Methods
A survey was carried out in two University Hospitals in Switzerland in autumn 2009 where the SAQ was distributed to a sample of 406 nurses and physicians in medical and surgical wards. Following the American Educational Research Association guidelines, we tested the questionnaire validity by levels of evidence: content validity, internal structure and relations to other variables. Confirmatory factor analysis was used to examine factor structure. Cronbach’s alphas and inter-item correlations were calculated to examine internal consistency reliability.
Results
A total of 319 questionnaires were completed representing an overall response rate of 78.6%. For three items, the item content validity index was <0.75. Confirmatory factor analysis showed acceptable model fit (RMSEA = 0.045; CFI = 0.944) for the six-factor model. Additional exploratory factor analysis could not identify a better factor model. SAQ factor scores showed positive correlations with the Safety Organizing Scale (r = .56 - .72). The SAQ German version showed moderate to strong internal consistency reliability indices (Cronbach alpha = .65 - .83).
Conclusions
The German language version of the SAQ demonstrated acceptable to good psychometric properties and therefore shows promise to be a sound instrument to measure patient safety climate in Swiss hospital wards. However, the low item content validity and large number of missing responses for several items suggest that improvements and adaptations in translation are required for select items, especially within the perception of management scale. Following these revisions, psychometric properties should reassessed in a randomly selected sample and hospitals and departments prior to use in Swiss hospital settings.
doi:10.1186/1472-6963-13-347
PMCID: PMC3846625  PMID: 24016183
Safety attitudes questionnaire (SAQ); Patient safety; Psychometrics; Swiss hospital setting
25.  The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions 
OBJECTIVE: To test the feasibility of creating a valid and reliable checklist with the following features: appropriate for assessing both randomised and non-randomised studies; provision of both an overall score for study quality and a profile of scores not only for the quality of reporting, internal validity (bias and confounding) and power, but also for external validity. DESIGN: A pilot version was first developed, based on epidemiological principles, reviews, and existing checklists for randomised studies. Face and content validity were assessed by three experienced reviewers and reliability was determined using two raters assessing 10 randomised and 10 non- randomised studies. Using different raters, the checklist was revised and tested for internal consistency (Kuder-Richardson 20), test-retest and inter-rater reliability (Spearman correlation coefficient and sign rank test; kappa statistics), criterion validity, and respondent burden. MAIN RESULTS: The performance of the checklist improved considerably after revision of a pilot version. The Quality Index had high internal consistency (KR-20: 0.89) as did the subscales apart from external validity (KR-20: 0.54). Test-retest (r 0.88) and inter-rater (r 0.75) reliability of the Quality Index were good. Reliability of the subscales varied from good (bias) to poor (external validity). The Quality Index correlated highly with an existing, established instrument for assessing randomised studies (r 0.90). There was little difference between its performance with non-randomised and with randomised studies. Raters took about 20 minutes to assess each paper (range 10 to 45 minutes). CONCLUSIONS: This study has shown that it is feasible to develop a checklist that can be used to assess the methodological quality not only of randomised controlled trials but also non-randomised studies. It has also shown that it is possible to produce a checklist that provides a profile of the paper, alerting reviewers to its particular methodological strengths and weaknesses. Further work is required to improve the checklist and the training of raters in the assessment of external validity.
 
PMCID: PMC1756728  PMID: 9764259

Results 1-25 (571257)