To further develop and validate previously published national prescribing appropriateness criteria to assist in identifying drug-related problems (DRPs) for commonly occurring medications and medical conditions in older (≥65 years old) Australians.
RAND/UCLA appropriateness method.
A panel of medication management experts were identified consisting of geriatricians/pharmacologists, clinical pharmacists and disease management advisors to organisations that produce Australian evidence-based therapeutic publications. This resulted in a round-one panel of 15 members, and a round-two panel of 12 members.
Main outcome measure
Agreement on all criteria.
Forty-eight prescribing criteria were rated. In the first rating round via email, there was disagreement regarding 17 of the criteria according to median panel ratings. During a face-to-face second round meeting, discussion resulted in retention of 25 criteria after amendments, agreement for 14 criteria with no changes required and deletion of 9 criteria. Two new criteria were added, resulting in a final validated list of 41 prescribing appropriateness criteria. Agreement after round two was reached for all 41 criteria, measured by median panel ratings and the amount of dispersion of panel ratings, based on the interpercentile range.
A set of 41 Australian prescribing appropriateness criteria were validated by an expert panel. Use of these criteria, together with clinical judgement and other medication review processes such as patient interview, is intended to assist in improving patient care by efficiently detecting potential DRPs related to commonly occurring medicines and medical conditions in older Australians. These criteria may also contribute to the medication management education of healthcare professionals.
drug-related problems; prescribing criteria; older patients; inappropriate drug use
Primary care clinicians initiate and oversee colorectal screening for their patients, but colonoscopy, a central component of screening programs, is usually performed by consultants. The accuracy and safety of colonoscopy varies among endoscopists, even those with mainstream training and certification. Therefore, it is a primary care responsibility to choose the best available colonoscopy services. A working group of the National Colorectal Cancer Roundtable identified a set of indicators that primary care clinicians can use to assess the quality of colonoscopy services. Quality measures are of actual performance, not training, specialty, or experience alone. The main elements of quality are a complete report, technical competence, and a safe setting for the procedure. We provide explicit criteria that primary care physicians can use when choosing a colonoscopist. Information on quality indicators will be increasingly available with quality improvement efforts within the colonoscopy community and growth in the use of electronic medical records.
primary care clinicians; colorectal screening; endoscopist; colonoscopist
To systematically develop a quality indicator (QI) set for systemic lupus erythematosus (SLE).
We used a validated process that combined available scientific evidence and expert consensus to develop a QI set for SLE. First, we extracted 20 candidate indicators from a systematic literature review of clinical practice guidelines pertaining to SLE. An advisory panel revised and augmented these candidate indicators, and through two rounds of voting, arrived at 25 QIs. These QIs advanced to the next phase of the project, in which we employed a modification of the RAND/UCLA Appropriateness Method. A systematic review of the literature was performed for each QI, linking the proposed process of care to potential improved health outcomes. After reviewing this scientific evidence, a second interdisciplinary expert panel convened to discuss the evidence and provide final ratings on the validity and feasibility of each QI.
The final expert panel rated 20 QIs as both valid and feasible. Areas covered include diagnosis, general preventive strategies (e.g. vaccinations, sun avoidance counseling, screening for cardiovascular disease), osteoporosis prevention and treatment, drug toxicity monitoring, renal disease, and reproductive health.
We employed a rigorous multi-step approach with systematic literature reviews and two expert panels to develop QIs for SLE. This new set of indicators provides an opportunity to assess health care quality in SLE, and represents an initial step toward the important goal of improving care in this patient population.
Administering medication to hospitalised infants and children is a complex process at high risk of error. Failure mode and effect analysis (FMEA) is a proactive tool used to analyse risks, identify failures before they happen and prioritise remedial measures. To examine the hazards associated with the process of drug delivery to children, we performed a proactive risk-assessment analysis.
Design and setting
Five multidisciplinary teams, representing different divisions of the paediatric department at Padua University Hospital, were trained to analyse the drug-delivery process, to identify possible causes of failures and their potential effects, to calculate a risk priority number (RPN) for each failure and plan changes in practices.
To identify higher-priority potential failure modes as defined by RPNs and planning changes in clinical practice to reduce the risk of patients harm and improve safety in the process of medication use in children.
In all, 37 higher-priority potential failure modes and 71 associated causes and effects were identified. The highest RPNs related (>48) mainly to errors in calculating drug doses and concentrations. Many of these failure modes were found in all the five units, suggesting the presence of common targets for improvement, particularly in enhancing the safety of prescription and preparation of endovenous drugs. The introductions of new activities in the revised process of administering drugs allowed reducing the high-risk failure modes of 60%.
FMEA is an effective proactive risk-assessment tool useful to aid multidisciplinary groups in understanding a process care and identifying errors that may occur, prioritising remedial interventions and possibly enhancing the safety of drug delivery in children.
Qualitative Research; Pediatrics; Drugs Administration
Swedish health care authorities use three key criteria to produce national guidelines for local priority setting: severity of the health condition, expected patient benefit, and cost-effectiveness of medical intervention. Priority setting in primary health care (PHC) has significant implications for health costs and outcomes in the health care system. Nevertheless, these guidelines have been implemented to a very limited degree in PHC. The objective of the study was to qualitatively assess how general practitioners (GPs) and nurses perceive the application of the three key priority-setting criteria.
Focus groups were held with GPs and nurses at primary health care centres, where the staff had a short period of experience in using the criteria for prioritising in their daily work.
The staff found the three key priority-setting criteria (severity, patient benefit, and cost-effectiveness) to be valuable for priority setting in PHC. However, when the criteria were applied in PHC, three additional dimensions were identified: 1) viewpoint (medical or patient's), 2) timeframe (now or later), and 3) evidence level (group or individual).
The three key priority-setting criteria were useful. Considering the three additional dimensions might enhance implementation of national guidelines in PHC and is probably a prerequisite for the criteria to be useful in priority setting for individual patients.
In the clinical management of patients at risk for or diagnosed with primary open-angle glaucoma (POAG), the aim of medical treatment is to reduce intraocular pressure (IOP) and then maintain it over time at a level that preserves both the structure and function of the optic nerve.
The objective of this report was to establish a consensus on the criteria that should be used to determine the characteristics of IOP-lowering medication.
Discussion was held among a panel of 12 physicians considered to be experts in glaucoma to develop a consensus on the criteria used by them to determine the characteristics of the IOP-lowering medication chosen for initial monotherapy and adjunctive treatment of ocular hypertension (OHT) or POAG. Consensus development combined available evidence and the impressions of these physicians regarding the clinical effectiveness of IOP-lowering medication for OHT and POAG. Once the panel identified the criteria, the order of priority and the relative importance of these criteria were then established in the setting of 3 risk categories (low, medium, and high) for a patient to experience significant visual disability from glaucoma over their expected life span.
The panel identified 5 criteria to determine the characteristics of IOP-lowering medication for OHT and POAG: IOP-lowering effect, systemic adverse events (AEs), ocular tolerability, compliance/administration, and cost of treatment. IOP-lowering effect was consistently ranked as the highest priority and cost as the lowest. The priority of compliance/administration did not vary by clinical situation. Systemic AEs and ocular tolerability were ranked as higher priorities in initial monotherapy than in adjunctive treatment and ranked lower as the risk for visual disability increased. The priority given to the criteria used to determine clinical effectiveness varied both with the risk for functional vision loss from glaucoma and whether initial monotherapy or adjunctive treatment was being considered.
Glaucoma treatment should be assessed with regard to the need not only to lower IOP but also to minimize systemic and ocular AEs, promote patient compliance, and minimize cost. The order of priority and relative importance given to these treatment criteria will vary as part of individualizing patient care.
consensus; criteria; intraocular pressure; glaucoma
Since 1991 the NHS has attempted to identify and prioritise its needs for research and development in a systematic manner. This has not been done before and there is little evidence on which to draw. Multidisciplinary expert groups have identified priorities in different topics using explicit criteria and after widespread consultation within the NHS and research community to identify pressing problems and opportunities for research. This paper focuses on a review completed in 1993 to identify research and development priorities for the NHS in relation to the interface between primary and secondary care. The review covered several recent developments which require evaluation. The authors describe the process used to identify research and development priorities in this complex subject and examine the strengths and weaknesses of the approach. This case study should help to stimulate a wider debate on methods of identifying priorities, particularly those using participatory approaches, in research and non-research contexts.
Introduction: Providing higher quality medical care to workers with occupationally associated carpal tunnel syndrome (CTS) may reduce disability, facilitate return to work, and lower the associated costs. Although many workers’ compensation systems have adopted treatment guidelines to reduce the overuse of unnecessary care, limited attention has been paid to ensuring that the care workers do receive is high quality. Further, guidelines are not designed to enable objective assessments of quality of care. This study sought to develop quality measures for the diagnostic evaluation and non-operative management of CTS, including managing occupational activities and functional limitations. Methods: Using a variation of the well-established RAND/UCLA Appropriateness Method, we developed draft quality measures using guidelines and literature reviews. Next, in a two-round modified-Delphi process, a multidisciplinary panel of 11 U.S. experts in CTS rated the measures on validity and feasibility. Results: Of 40 draft measures, experts rated 31 (78%) valid and feasible. Nine measures pertained to diagnostic evaluation, such as assessing symptoms, signs, and risk factors. Eleven pertain to non-operative treatments, such as the use of splints, steroid injections, and medications. Eleven others address assessing the association between symptoms and work, managing occupational activities, and accommodating functional limitations. Conclusions: These measures will complement existing treatment guidelines by enabling providers, payers, policymakers, and researchers to assess quality of care for CTS in an objective, structured manner. Given the characteristics of previous measures developed with these methods, greater adherence to these measures will probably lead to improved patient outcomes at a population level.
Carpal tunnel syndrome; Health care quality assurance; Standards
Improving palliative care management in acute hospital settings has been identified as a priority internationally. The aim of this study was to establish the proportion of inpatients within one acute hospital in New Zealand who meet prognostic criteria for palliative care need and explore key aspects of their management.
A prospective survey of adult hospital inpatients (n = 501) was undertaken. Case notes were examined for evidence that the patient might be in their last year of life according to Gold Standards Framework (GSF) prognostic indicator criteria. For patients who met GSF criteria, clinical and socio-demographic information were recorded.
Ninety-nine inpatients met GSF criteria, representing 19.8% of the total census population. The patients’ average age was 70 years; 47% had a primary diagnosis of cancer. Two thirds had died within 6 months of their admission. Seventy-eight of the 99 cases demonstrated evidence that a palliative approach to care had been adopted; however documentation of discussion about goals of care was very limited and only one patient had evidence of an advance care plan.
One fifth of hospital inpatients met criteria for palliative care need, the majority of whom were aged >70 years. Whilst over three quarters were concluded to be receiving care in line with a palliative care approach, very little documented evidence of discussion with patients and families regarding end of life issues was evident. Future research needs to explore how best to support ‘generalist’ palliative care providers in initiating, and appropriately recording, such discussions.
Palliative care; End of life; Census; Healthcare; Acute; Hospital
Cesarean section rates are increasing worldwide, and a rapid increase has been observed in Iran. Disagreement exists between clinicians about when to use cesarean section. We aimed to identify the appropriateness criteria for the use of cesarean section in Iran.
A consensus development study using a modified version of the RAND Appropriateness Method (RAM). We generated scenarios from valid clinical guidelines and expert opinions. A panel of experts participated in consensus development: first round via mail (12 members), second round face-to-face (9 members). We followed the RAM recommendations for the development of the scenario lists, rating scales, and statistical analyses.
294 scenarios relevant to cesarean section were identified. 191 scenarios were considered appropriate, of which 125 scenarios were agreed upon. The panel found cesarean inappropriate for 21% of scenarios, and 'equivocal' for 14% of scenarios.
RAM is useful for identifying stakeholder views in settings with limited resources. The participants' views on appropriateness of certain indications differed with available evidence. A large number of scenarios without agreement may partly explain why it has been difficult to curb the growth in cesarean section rate.
Aim: To apply in practice a series of validated indicators for preventable drug related morbidity (PDRM).
Design: A pilot study to identify retrospectively potential PDRM events over a 2 year 3 month time frame using the MIQUEST computer software program.
Subjects and setting: The electronic patient record of all patients aged 18 years and over in nine English general practices.
Outcome measures: The number of potential PDRM events identified, as defined by the indicators.
Results: Five hundred and seven potential PDRM events were identified from 49 658 electronic patient records, giving an overall incidence of 1.0%. A small number of the indicators (n = 4) accounted for approximately 60% of the events, while for many indicators few events were identified. The most common events related to the use of non-steroidal anti-inflammatory drugs in patients with congestive heart failure or hypertension, lack of monitoring in patients prescribed angiotensin converting enzyme inhibitors, and the use of hypnotic-anxiolytic agents.
Conclusions: A small number of indicators contributed to the majority of the PDRM events. Interrogation of electronic patient records in primary care using computerised queries shows potential for detecting PDRM.
This study aimed at evaluating face and content validity, feasibility and reliability of process quality indicators developed previously in the United States or other countries. The indicators can be used to evaluate care and services for vulnerable older adults affected by cognitive impairment or dementia within an integrated service system in Quebec, Canada.
A total of 33 clinical experts from three major urban centres in Quebec formed a panel representing two medical specialties (family medicine, geriatrics) and seven health or social services specialties (nursing, occupational therapy, psychology, neuropsychology, pharmacy, nutrition, social work), from primary or secondary levels of care, including long-term care. A modified version of the RAND®/University of California at Los Angeles (UCLA) appropriateness method, a two-round Delphi panel, was used to assess face and content validity of process quality indicators. The appropriateness of indicators was evaluated according to a) agreement of the panel with three criteria, defined as a median rating of 7–9 on a nine-point rating scale, and b) agreement among panellists, judged by the statistical measure of the interpercentile range adjusted for symmetry. Feasibility of quality assessment and reliability of appropriate indicators were then evaluated within a pilot study on 29 patients affected by cognitive impairment or dementia. For measurable indicators the inter-observer reliability was calculated with the Kappa statistic.
Initially, 82 indicators for care of vulnerable older adults with cognitive impairment or dementia were submitted to the panellists. Of those, 72 (88%) were accepted after two rounds. Among 29 patients for whom medical files of the preceding two years were evaluated, 63 (88%) of these indicators were considered applicable at least once, for at least one patient. Only 22 indicators were considered applicable at least once for ten or more out of 29 patients. Four indicators could be measured with the help of a validated questionnaire on patient satisfaction. Inter-observer reliability was moderate (Kappa = 0.57).
A multidisciplinary panel of experts judged a large majority of the initial indicators valid for use in integrated care systems for vulnerable older adults in Quebec, Canada. Most of these indicators can be measured using patient files or patient or caregiver interviews and reliability of assessment from patient-files is moderate.
Determining whether persons with multiple sclerosis (MS) receive appropriate,
comprehensive healthcare requires tools for measuring quality. The objective of
this study was to develop quality indicators for the care of persons with MS. We
used a modified version of the RAND/UCLA Appropriateness Method in a two-stage
process to identify relevant MS care domains and to assess the validity of
indicators within high-ranking care domains. Based on a literature review,
interviews with persons with MS, and discussions with MS providers, 25 MS
symptom domains and 14 general health domains of MS care were identified. A
multidisciplinary panel of 15 stakeholders of MS care, including 4 persons with
MS, rated these 39 domains in a two-round modified Delphi process. The research
team performed an expanded literature review for 26 highly ranked domains to
draft 86 MS care indicators. Through another two-round modified Delphi process,
a second panel of 18 stakeholders rated these indicators using a nine-point
response scale. Indicators with a median rating in the highest tertile were
considered valid. Among the most highly rated MS care domains were
appropriateness and timeliness of the diagnostic work-up, bladder dysfunction,
cognition dysfunction, depression, disease-modifying agent usage, fatigue,
integration of care, and spasticity. Of the 86 preliminary indicators, 76 were
rated highly enough to meet predetermined thresholds for validity. Following a
widely accepted methodology, we developed a comprehensive set of quality
indicators for MS care that can be used to assess quality of care and guide the
design of interventions to improve care among persons with MS.
health services research; multiple sclerosis; outcome research; quality indicators
OBJECTIVE: To develop review criteria to assess the quality of care for
three major chronic diseases: adult asthma, stable angina, and non- insulin
dependent diabetes mellitus. SUBJECTS AND METHODS: Modified panel process
based upon the RAND/UCLA (University College of Los Angeles)
appropriateness method. Three multiprofessional panels made up of general
practitioners, hospital specialists, and practice nurses. RESULTS: The
RAND/UCLA appropriateness method of augmenting evidence with expert opinion
was used to develop criteria for the care of the three conditions. Of those
aspects of care which were rated as necessary by the panels, only 26% (16%
asthma, 10% non-insulin dependent diabetes, 40% angina) were subsequently
rated by the panels as being based on strong scientific evidence.
CONCLUSION: The results show the importance of a systematic approach to
combining evidence with expert opinion to develop review criteria for
assessing the quality of three chronic diseases in general practice. The
evidence base for the criteria was often incomplete, and explicit methods
need to be used to combine evidence with expert opinion where evidence is
Background: Measurement of the quality of health care is essential for quality improvement, and patients are an underused source of data about quality of care. We describe the adaptation of a set of USA quality indicators for use in patient interview surveys in England, to measure the extent to which older patients receive a broad range of effective healthcare interventions in both primary and secondary care.
Method: One hundred and nineteen quality indicators covering 16 clinical areas, based on a set of indicators for the care of vulnerable elderly patients in the USA, were reviewed by a panel of 10 clinical experts in England. A modified version of the RAND/UCLA appropriateness method was used and panel members were supplied with literature reviews summarising the evidence base for each quality indicator. The indicators were sent for comment before the panel meeting to UK charitable organisations for older people.
Results: The panel rated 102 of the 119 indicators (86%) as valid for use in England; 17 (14%) were rejected as invalid. All 58 indicators about treatment or continuity and follow up were rated as valid compared with just over half (13 of 24) of the indicators about screening.
Conclusions: These 102 indicators are suitable for use in patient interview surveys, including the English Longitudinal Study of Ageing (ELSA). The systematic measurement of quality of care at the population level and identification of gaps in quality is essential for quality improvement. There is potential for transfer of quality indicators between countries, at least for the health care of older people.
Internationally, many health care interventions were diffused prior to the standard use of assessments of safety, effectiveness and cost-effectiveness. Disinvestment from ineffective or inappropriately applied practices is a growing priority for health care systems for reasons of improved quality of care and sustainability of resource allocation. In this paper we examine key challenges for disinvestment from these interventions and explore potential policy-related avenues to advance a disinvestment agenda.
We examine five key challenges in the area of policy driven disinvestment: 1) lack of resources to support disinvestment policy mechanisms; 2) lack of reliable administrative mechanisms to identify and prioritise technologies and/or practices with uncertain clinical and cost-effectiveness; 3) political, clinical and social challenges to removing an established technology or practice; 4) lack of published studies with evidence demonstrating that existing technologies/practices provide little or no benefit (highlighting complexity of design) and; 5) inadequate resources to support a research agenda to advance disinvestment methods. Partnerships are required to involve government, professional colleges and relevant stakeholder groups to put disinvestment on the agenda. Such partnerships could foster awareness raising, collaboration and improved health outcome data generation and reporting. Dedicated funds and distinct processes could be established within the Medical Services Advisory Committee and Pharmaceutical Benefits Advisory Committee to, a) identify technologies and practices for which there is relative uncertainty that could be the basis for disinvestment analysis, and b) conduct disinvestment assessments of selected item(s) to address existing practices in an analogous manner to the current focus on new and emerging technology. Finally, dedicated funding and cross-disciplinary collaboration is necessary to build health services and policy research capacity, with a focus on advancing disinvestment research methodologies and decision support tools.
The potential over-utilisation of less than effective clinical practices and the potential under-utilisation of effective clinical practices not only result in less than optimal care but also fragmented, inefficient and unsustainable resource allocation. Systematic policy approaches to disinvestment will improve equity, efficiency, quality and safety of care, as well as sustainability of resource allocation.
OBJECTIVE—To evaluate the appropriateness of referral following coronary angiography in Sweden.
DESIGN—Prospective survey and review of medical records.
PATIENTS—Consecutive series of 2767 patients who underwent coronary angiography in Sweden between May 1994 and January 1995 and were considered for coronary revascularisation.
MAIN OUTCOME MEASURES—Percentage of patients referred for coronary artery bypass graft surgery (CABG) and percutaneous transluminal coronary angioplasty (PTCA) for indications that were judged necessary, appropriate, uncertain, and inappropriate by a multispecialty Swedish national expert panel using the RAND/University of California Los Angeles (UCLA) appropriateness method, and the percentage of patients referred for continued medical management who met necessity criteria for revascularisation.
RESULTS—Half the patients were referred for CABG, 25% for PTCA, and 25% for continued medical therapy. CABG was judged appropriate or necessary for 78% of patients, uncertain for 12% and inappropriate for 10%. For PTCA the figures were 32%, 30% and 38%, respectively. Two factors contributed to the high inappropriate rate. Many of these patients did not have "significant" coronary artery disease (although all had at least one stenosis > 50%) or they were treated with less than "optimal" medical therapy. While 96% of patients who met necessity criteria for revascularisation were appropriately referred for revascularisation, 4% were referred for continued medical therapy.
CONCLUSIONS—Using the RAND/UCLA appropriateness method and the definitions agreed to by the expert panel, which may be considered conservative today, it was found that 19% of Swedish patients were referred for coronary revascularisation judged inappropriate. Since some cardiovascular procedures evolve rapidly, the proportion of patients referred for inappropriate indications today remains unknown. Nevertheless, physicians should actively identify those patients who will and will not benefit from coronary revascularisation and ensure that they are appropriately treated.
Keywords: coronary artery bypass graft surgery; angioplasty; quality of health care; health policy
To examine the effects of a medication review project by geriatricians and geriatric medicine fellows on polypharmacy in a teaching nursing home.
Quality Improvement Intervention Study
Long-term care facility in Honolulu, HI
Seventy-four patients with the Minimum Data Set (MDS) quality indicator (QI) criteria of polypharmacy (nine or more medications).
Geriatric Medicine Fellows and Faculty reviewed each patient’s medication list, consulted the updated Beers Criteria and Epocrates online drug-drug interaction program, and recommended medication changes to the patients’ primary care physicians.
Descriptive statistics, including means, standard deviations, and sums of variables were obtained for the number of medications in the following categories: total number, scheduled, Pro re nata (PRN), high risk, contraindicated, with potential drug-drug interactions, and with no indication.
Out of 160 patients residing in a nursing home, 74 were on nine or more medications. After the intervention, the mean number of medications per patient in the following categories decreased significantly: total number (16.64 to 15.54, p<.001), scheduled (11.3 to 10.99, p<.001), PRN (5.33 to 4.56, p<.001), high risk (0.94 to 0.73, p<.001), contraindicated (0.29 to 0.13, p=.004), with potential drug-drug interactions (6.1 to 4.83, p<.001), and with no indication (3.34 to 3.29, p=.045).
Polypharmacy in long-term care is prevalent and can lead to increased adverse effects and potentially inappropriate prescriptions. This study demonstrates an effective geriatrician-led intervention that both reduced polypharmacy and provided core competency training for geriatric medicine fellows.
polypharmacy; long-term care; nursing home
Tens of millions of patients worldwide suffer disabling injuries or death every year due to unsafe medical care. Nonetheless, there is a scarcity of research evidence on how to tackle this global health priority. The shortage of trained researchers is a major limitation, particularly in developing and transitional countries.
As a first step to strengthen capacity in this area, the authors developed a set of internationally agreed core competencies for patient safety research worldwide.
A multistage process involved developing an initial framework, reviewing the existing literature relating to competencies in patient safety research, conducting a series of consultations with potential end users and international experts in the field from over 35 countries and finally convening a global consensus conference.
An initial draft list of competencies was grouped into three themes: patient safety, research methods and knowledge translation. The competencies were considered by the WHO Patient Safety task force, by potential end users in developing and transitional countries and by international experts in the field to be relevant, comprehensive, clear, easily adaptable to local contexts and useful for training patient safety researchers internationally.
Reducing patient harm worldwide will require long-term sustained efforts to build capacity to enable practical research that addresses local problems and improves patient safety. The first edition of Competencies for Patient Safety Researchers is proposed by WHO Patient Safety as a foundation for strengthening research capacity by guiding the development of training programmes for researchers in the area of patient safety, particularly in developing and transitional countries, where such research is urgently needed.
Competencies; patient safety; research; capacity building; developing countries; health policy; health professions education; training
Currently there is no framework for those involved in the identification, evaluation and prioritisation of new diagnostic technologies. Therefore we aimed to develop prioritisation criteria for the assessment of new diagnostic technologies, by gaining international consensus on not only which criteria should be used, but also their relative importance.
A two-round Delphi process was used to generate consensus amongst an international panel of twenty-six experts on priority criteria for diagnostic health technology assessment. Participants represented a range of health care and related professions, including government, industry, health services and academia.
Based on the responses to the first questionnaire 18 criteria were placed into three categories: high, intermediate and moderate priority. For 16 of the 18 criteria, agreement with the categorisation of the criteria into the high, intermediate and moderate categories was high at ≥ 70% (10 had agreement ≥ 80%). A further questionnaire and panel discussion reduced the criteria to 16 and two categories; seven were classified as high priority and nine intermediate.
This study proposes an objective structure of prioritisation criteria to use when assessing new diagnostic technologies, based on an expert consensus process. The value of these criteria is that no one single component should be used as the decisive driver for prioritisation of new diagnostic technologies for adoption in healthcare settings. Future studies should be directed at establishing the value of these prioritisation criteria across a range of healthcare settings.
Consensus development techniques were used in the late 1980s to create explicit criteria for the appropriateness of cataract extraction. We developed a new appropriateness of indications tool for cataract following the RAND method. We tested the validity of our panel results.
Criteria were developed using a modified Delphi panel judgment process. A panel of 12 ophthalmologists was assembled. Ratings were analyzed regarding the level of agreement among panelists. We studied the influence of all variables on the final panel score using linear and logistic regression models. The explicit criteria developed were summarized by classification and regression tree analysis.
Of the 765 indications evaluated by the main panel in the second round, 32.9% were found appropriate, 30.1% uncertain, and 37% inappropriate. Agreement was found in 53% of the indications and disagreement in 0.9%. Seven variables were considered to create the indications and divided into three groups: simple cataract, with diabetic retinopathy, or with other ocular pathologies. The preoperative visual acuity in the cataractous eye and visual function were the variables that best explained the panel scoring. The panel results were synthesized and presented in three decision trees. Misclassification error in the decision trees, as compared with the panel original criteria, was 5.3%.
The parameters tested showed acceptable validity for an evaluation tool. These results support the use of this indication algorithm as a screening tool for assessing the appropriateness of cataract extraction in field studies and for the development of practice guidelines.
To determine whether feedback from attending physicians to residents about outpatient medical records improves chart documentation and quality of care.
Cross-sectional study with repeated measures.
Primary care internal medicine clinic at a metropolitan community hospital.
Fifteen interns and 20 residents.
Attending physicians reviewed at least two charts for each resident on three occasions about 4 months apart and then discussed their findings with the residents.
MEASUREMENTS AND MAIN RESULTS
Explicit criteria defined the extent of chart documentation and the comprehensiveness of care delivery. Attending physicians also made a subjective assessment of the overall quality of care. All results were converted to 0-to-1 scales. From the first to the third period, chart documentation increased from 0.60 to 0.86 (p < .001), but there were no significant changes in the delivery of care or in the subjective assessments of the overall quality of care.
Both review of residents' outpatient medical records and periodic feedback from attending physicians improve how well medical housestaff document care in the chart.
documentation; quality of care; feedback; outpatients
Objective: To evaluate the transferability of primary care quality indicators by comparing indicators for common clinical problems developed using the same method in the UK and the USA.
Method: Quality indicators developed in the USA for a range of common conditions using the RAND-UCLA appropriateness method were applied to 19 common primary care conditions in the UK. The US indicators for the selected conditions were used as a starting point, but the literature reviews were updated and panels of UK primary care practitioners were convened to develop quality indicators applicable to British general practice.
Results: Of 174 indicators covering 18 conditions in the US set for which a direct comparison could be made, 98 (56.3%) had indicators in the UK set which were exactly or nearly equivalent. Some of the differences may have related to differences in the process of developing the indicators, but many appeared to relate to differences in clinical practice or norms of professional behaviour in the two countries. There was a small but non-significant relationship between the strength of evidence for an indicator and the probability of it appearing in both sets of indicators.
Conclusion: There are considerable benefits in using work from other settings in developing measures of quality of care. However, indicators cannot simply be transferred directly between countries without an intermediate process to allow for variation in professional culture or clinical practice.
Making health care safer is a key policy priority worldwide. In specialty training, medical educators may unintentionally impact on patient safety e.g. through failures of supervision; providing limited feedback on performance; and letting poorly developed behaviours continue unchecked. Doctors-in-training are also known to be susceptible to medical error. Ensuring that all essential educational issues are addressed during training is problematic given the scale of the tasks to be undertaken. Human error and the reliability of local systems may increase the risk of safety-critical topics being inadequately covered. However adherence to a checklist reminder may improve the reliability of task delivery and maximise harm reduction. We aimed to prioritise the most safety-critical issues to be addressed in the first 12-weeks of specialty training in the general practice environment and validate a related checklist reminder.
We used mixed methods with different groups of GP educators (n = 127) and specialty trainees (n = 9) in two Scottish regions to prioritise, develop and validate checklist content. Generation and refinement of checklist themes and items were undertaken on an iterative basis using a range of methods including small group work in dedicated workshops; a modified-Delphi process; and telephone interviews. The relevance of potential checklist items was rated using a 4-point scale content validity index to inform final inclusion.
14 themes (e.g. prescribing safely; dealing with medical emergency; implications of poor record keeping; and effective & safe communication) and 47 related items (e.g. how to safety-net face-to-face or over the telephone; knowledge of practice systems for results handling; recognition of harm in children) were judged to be essential safety-critical educational issues to be covered. The mean content validity index ratio was 0.98.
A checklist was developed and validated for educational supervisors to assist in the reliable delivery of safety-critical educational issues in the opening 12-week period of training, and aligned with national curriculum competencies. The tool can also be adapted for use as a self-assessment instrument by trainees to guide patient safety-related learning needs. Dissemination and implementation of the checklist and self-rating scale are proceeding on a national, voluntary basis with plans to evaluate its feasibility and educational impact.
Jargon is a barrier to effective patient-physician communication, especially when health literacy is low or the topic is complicated. Jargon is addressed by medical schools and residency programs, but reducing jargon usage by the many physicians already in practice may require the population-scale methods used in Quality Improvement.
To assess the amount of jargon used and explained during discussions about prostate or breast cancer screening. Effective communication is recommended before screening for prostate or breast cancer because of the large number of false-positive results and the possible complications from evaluation or treatment.
Primary care internal medicine residents.
Transcripts of 86 conversations between residents and standardized patients were abstracted using an explicit-criteria data dictionary. Time lag from jargon words to explanations was measured using “statements,” each of which contains one subject and one predicate.
Duplicate abstraction revealed reliability κ = 0.92. The average number of unique jargon words per transcript was 19.6 (SD = 6.1); the total jargon count was 53.6 (SD = 27.2). There was an average of 4.5 jargon-explanations per transcript (SD = 2.3). The ratio of explained to total jargon was 0.15. When jargon was explained, the average time lag from the first usage to the explanation was 8.4 statements (SD = 13.4).
The large number of jargon words and low number of explanations suggest that many patients may not understand counseling about cancer screening tests. Educational programs and faculty development courses should continue to discourage jargon usage. The methods presented here may be useful for feedback and quality improvement efforts.