Search tips
Search criteria

Results 1-25 (44)

Clipboard (0)

Select a Filter Below

Year of Publication
more »
1.  Reporting of context and implementation in studies of global health interventions: a pilot study 
There is an increasing push for ‘evidence-based’ decision making in global health policy circles. However, at present there are no agreed upon standards or guidelines for how to evaluate evidence in global health. Recent evaluations of existing evidence frameworks that could serve such a purpose have identified details of program context and project implementation as missing components needed to inform policy. We performed a pilot study to assess the current state of reporting of context and implementation in studies of global health interventions.
We identified three existing criteria sets for implementation reporting and selected from them 10 criteria potentially relevant to the needs of policy makers in global health contexts. We applied these 10 criteria to 15 articles included in the evidence base for three global health interventions chosen to represent a diverse set of advocated global health programs or interventions: household water chlorination, prevention of mother-to-child transmission of HIV, and lay community health workers to reduce child mortality. We used a good-fair-poor/none scale for the ratings.
The proportion of criteria for which reporting was poor/none ranged from 11% to 54% with an average of 30%. Eight articles had ‘good’ or ‘fair’ documentation for greater than 75% of criteria, while five articles had ‘poor or none’ documentation for 50% of criteria or more. Examples of good reporting were identified.
Reporting of context and implementation information in studies of global health interventions is mostly fair or poor, and highly variable. The idiosyncratic variability in reporting indicates that global health investigators need more guidance about what aspects of context and implementation to measure and how to report them. This lack of context and implementation information is a major gap in the evidence needed by global health policy makers to reach decisions.
PMCID: PMC4043974  PMID: 24886201
2.  When Comorbidity, Aging, and Complexity of Primary Care Meet: Development and Validation of the Geriatric CompleXity of Care Index 
To develop and validate the Geriatric CompleXity of Care Index (GXI), a comorbidity index of medical, geriatric, and psychosocial conditions that addresses disease severity and intensity of ambulatory care for older adults with chronic conditions.
Development phase: variable selection and rating by clinician panel. Validation phase: medical record review and secondary data analysis.
Assessing the Care of Vulnerable Elders-2 study.
Six hundred forty-four older (≥75) individuals receiving ambulatory care.
Development: 32 conditions categorized according to severity, resulting in 117 GXI variables. A panel of clinicians rated each GXI variable with respect to the added difficulty of providing primary care for an individual with that condition. Validation: Modified versions of previously validated comorbidity measures (simple count, Charlson, Medicare Hierarchical Condition Category), longitudinal clinical outcomes (functional decline, survival), intensity of ambulatory care (primary, specialty care visits, polypharmacy, number of eligible quality indicators (NQI)) over 1 year of care.
The most-morbid individuals (according to quintiles of GXI) had more visits (7.0 vs 3.7 primary care, 6.2 vs 2.4 specialist), polypharmacy (14.3% vs 0% had ≥14 medications), and greater NQI (33 vs 25) than the least-morbid individuals. Of the four comorbidity measures, the GXI was the strongest predictor of primary care visits, polypharmacy, and NQI (p<.001, controlling for age, sex, function-based vulnerability).
Older adults with complex care needs, as measured by the GXI, have healthcare needs above what previously employed comorbidity measures captured. Healthcare systems could use the GXI to identify the most complex elderly adults and appropriately reimburse primary providers caring for older adults with the most complex care needs for providing additional visits and coordination of care.
PMCID: PMC3698956  PMID: 23581912
ambulatory care; utilization; comorbidity
4.  Assessment of a method to detect signals for updating systematic reviews 
Systematic Reviews  2014;3:13.
Systematic reviews are a cornerstone of evidence-based medicine but are useful only if up-to-date. Methods for detecting signals of when a systematic review needs updating have face validity, but no proposed method has had an assessment of predictive validity performed.
The AHRQ Comparative Effectiveness Review program had produced 13 comparative effectiveness reviews (CERs), a subcategory of systematic reviews, by 2009, 11 of which were assessed in 2009 using a surveillance system to determine the degree to which individual conclusions were out of date and to assign a priority for updating each report. Four CERs were judged to be a high priority for updating, four CERs were judged to be medium priority for updating, and three CERs were judged to be low priority for updating. AHRQ then commissioned full update reviews for 9 of these 11 CERs. Where possible, we matched the original conclusions with their corresponding conclusions in the update reports, and compared the congruence between these pairs with our original predictions about which conclusions in each CER remained valid. We then classified the concordance of each pair as good, fair, or poor. We also made a summary determination of the priority for updating each CER based on the actual changes in conclusions in the updated report, and compared these determinations with the earlier assessments of priority.
The 9 CERs included 149 individual conclusions, 84% with matches in the update reports. Across reports, 83% of matched conclusions had good concordance, and 99% had good or fair concordance. The one instance of poor concordance was partially attributable to the publication of new evidence after the surveillance signal searches had been done. Both CERs originally judged as being low priority for updating had no substantive changes to their conclusions in the actual updated report. The agreement on overall priority for updating between prediction and actual changes to conclusions was Kappa = 0.74.
These results provide some support for the validity of a surveillance system for detecting signals indicating when a systematic review needs updating.
PMCID: PMC3937021  PMID: 24529068
Methods; Systematic reviews; Updating
International urogynecology journal  2013;24(12):2039-2047.
A paucity of data exists addressing the quality of care provided to women with pelvic organ prolapse (POP). We sought to develop a means to measure this quality through the development of quality-of-care indicators (QIs).
QIs were modeled after those previously described in the Assessing the Care of Vulnerable Elders (ACOVE) project. The indicators were then presented to a panel of nine experts. Using the RAND Appropriateness Method, we analyzed each indicator’s preliminary rankings. A forum was then held in which each indicator was thoroughly discussed by the panelists as a group, after which panelists individually re-rated the indicators. QIs with median scores of at least seven were considered valid.
QIs were developed that addressed screening, diagnosis, work-up, and both nonsurgical and surgical management. Areas of controversy included whether screening should be performed to identify prolapse, whether pessary users should undergo a vaginal exam by a health professional every six months versus annually, and whether a colpocleisis should be offered to older women planning to undergo surgery for POP. Fourteen of 21 potential indicators were rated as valid for pelvic organ prolapse (median score ≥ 7).
We developed and rated fourteen potential quality indicators for the care of women with POP. Once these QIs are tested for feasibility they can be used on a larger scale to measure and compare the care provided to women with prolapse in different clinical settings.
PMCID: PMC3890317  PMID: 23644812
Delphi Method; RAND Appropriateness method; pelvic floor disorders
Neurourology and urodynamics  2013;32(8):10.1002/nau.22353.
To develop a means to measure the quality of care provided to women treated for urinary incontinence (UI) through the development of quality-of-care indicators (QIs).
We performed an extensive literature review to develop a set of potential quality indicators for the management of urinary incontinence. QIs were modeled after those previously described in the Assessing the Care of Vulnerable Elders (ACOVE) project. Nine experts ranked the indicators on a nine-point scale for both validity and feasibility. We analyzed preliminary rankings of each indicator using the RAND Appropriateness Method. A forum was then held in which each indicator was thoroughly discussed by the panelists as a group, after which the indicators were rated a second time individually using the same nine-point scale.
QIs were developed that addressed screening, diagnosis, work-up, and both non-surgical and surgical management. Areas of controversy included whether routine screening for incontinence should be performed, whether urodynamics should be performed before non-surgical management is initiated, and whether cystoscopy should be part of the pre-operative work-up of uncomplicated stress incontinence. Following the expert panel discussion, 27 of 40 potential indicators were determined to be valid for UI with a median score of at least seven on a nine-point scale.
We identified 27 quality indicators for the care of women with UI. Once these QIs are pilot-tested for feasibility, they will be applied on a larger scale to measure the quality of care provided to women with UI in the United States.
PMCID: PMC3857939  PMID: 24105879
Quality Indicators; Outcomes; RAND Appropriateness Method; Stress Urinary Incontinence; Urge Urinary Incontinence
7.  How can we recognize continuous quality improvement? 
Continuous quality improvement (CQI) methods are foundational approaches to improving healthcare delivery. Publications using the term CQI, however, are methodologically heterogeneous, and labels other than CQI are used to signify relevant approaches. Standards for identifying the use of CQI based on its key methodological features could enable more effective learning across quality improvement (QI) efforts. The objective was to identify essential methodological features for recognizing CQI.
Previous work with a 12-member international expert panel identified reliably abstracted CQI methodological features. We tested which features met rigorous a priori standards as essential features of CQI using a three-phase online modified-Delphi process.
Primarily United States and Canada.
119 QI experts randomly assigned into four on-line panels.
Participants rated CQI features and discussed their answers using online, anonymous and asynchronous discussion boards. We analyzed ratings quantitatively and discussion threads qualitatively.
Main outcome measure(s)
Panel consensus on definitional CQI features.
Seventy-nine (66%) panelists completed the process. Thirty-three completers self-identified as QI researchers, 18 as QI practitioners and 28 as both equally. The features ‘systematic data guided activities,’ ‘designing with local conditions in mind’ and ‘iterative development and testing’ met a priori standards as essential CQI features. Qualitative analyses showed cross-cutting themes focused on differences between QI and CQI.
We found consensus among a broad group of CQI researchers and practitioners on three features as essential for identifying QI work more specifically as ‘CQI.’ All three features are needed as a minimum standard for recognizing CQI methods.
PMCID: PMC3914565  PMID: 24311732
continuous quality improvement; quality improvement; consultants; health care organization
8.  A surveillance system to assess the need for updating systematic reviews 
Systematic Reviews  2013;2:104.
Systematic reviews (SRs) can become outdated as new evidence emerges over time. Organizations that produce SRs need a surveillance method to determine when reviews are likely to require updating. This report describes the development and initial results of a surveillance system to assess SRs produced by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program.
Twenty-four SRs were assessed using existing methods that incorporate limited literature searches, expert opinion, and quantitative methods for the presence of signals triggering the need for updating. The system was designed to begin surveillance six months after the release of the original review, and thenceforth every six months for any review not classified as being a high priority for updating. The outcome of each round of surveillance was a classification of the SR as being low, medium or high priority for updating.
Twenty-four SRs underwent surveillance at least once, and ten underwent surveillance a second time during the 18 months of the program. Two SRs were classified as high, five as medium, and 17 as low priority for updating. The time lapse between the searches conducted for the original reports and the updated searches (search time lapse - STL) ranged from 11 months to 62 months: The STL for the high priority reports were 29 months and 54 months; those for medium priority reports ranged from 19 to 62 months; and those for low priority reports ranged from 11 to 33 months. Neither the STL nor the number of new relevant articles was perfectly associated with a signal for updating. Challenges of implementing the surveillance system included determining what constituted the actual conclusions of an SR that required assessing; and sometimes poor response rates of experts.
In this system of regular surveillance of 24 systematic reviews on a variety of clinical interventions produced by a leading organization, about 70% of reviews were determined to have a low priority for updating. Evidence suggests that the time period for surveillance is yearly rather than the six months used in this project.
PMCID: PMC3874670  PMID: 24225065
Systematic review; Updating; Surveillance
9.  Happy birthday: we are one year old 
Systematic Reviews  2013;2:61.
PMCID: PMC3751830
10.  A Comparison of Frameworks Evaluating Evidence for Global Health Interventions 
PLoS Medicine  2013;10(7):e1001469.
Jill Luoto and colleagues apply different frameworks to the same body of evidence for three advocated global health interventions and compare the ratings and policy recommendations resulting from each.
Please see later in the article for the Editors' Summary
PMCID: PMC3706307  PMID: 23874159
11.  Does Better Quality of Care for Falls and Urinary Incontinence Result in Better Patient-Reported Outcomes? 
Validation of process-of-care measures includes testing for a link with outcomes. We aimed to determine whether delivery of better quality of care for urinary incontinence (UI) and falls is associated with improved patient-reported outcomes.
Study design
Retrospective cohort study of older (age ≥ 75) ambulatory care participants in Assessing Care of Vulnerable Elders-2 (ACOVE-2) study, who screened positive for UI (n=133) and/or falls/fear of falling (n=328).
We measured composite quality scores (% quality indicators [QIs] passed per patient) and change in Incontinence Quality of Life (IQOL, range 0–100) scores or Falls Efficacy Scale (FES, range 10–40) before and after care was delivered (mean 10 months). Because eligibility for falls treatment QIs was dependent on the physician performing a physical exam, we calculated an alternative “Common Pathway” quality indicator (CPQI) score that assigned a failing score for falls treatment to unexamined patients.
Each 10% increment in receipt of recommended care for UI was associated with a 1.4 point improvement in IQOL score (p=.01). Falls quality was not related to FES score using the original composite score; however the CPQI score was related to FES (+.4 point FES per 10% increment in falls quality, p=.01).
Better quality of care for falls and UI was associated with measurable improvement in patient-reported outcomes in less than one year. The connection between process and outcome required consideration of the interdependence between diagnosis and treatment in the falls QIs. The process-outcome link demonstrated for UI and falls underscores the importance of improving care in these areas.
PMCID: PMC3654865  PMID: 21806560
12.  Is Patient-Perceived Severity of a Geriatric Condition Related to Better Quality of Care? 
Medical care  2011;49(1):101-107.
Care for falls and urinary incontinence (UI) among older patients is inadequate. One possible explanation is that physicians provide less recommended care to patients who are not as concerned about their falls and UI.
To test whether patient-reported severity for two geriatric conditions, falls and UI, is associated with quality of care.
Research Design
Prospective cohort study of elders with falls and/or fear of falling (n=384) and UI (n=163).
Participants in the Assessing Care of Vulnerable Elders-2 Study (2002–3), which evaluated an intervention to improve the care for falls and UI among older (age ≥75) ambulatory care patients with falls/fear of falling or UI.
Falls Efficacy Scale (FES) and the Incontinence Quality of Life (IQOL) surveys measured at baseline, quality of care measured by a 13-month medical record abstraction.
There was a small difference in falls quality scores across the range of FES, with greater patient-perceived falls severity associated with better odds of passing falls quality indicators (OR 1.11 (95% CI 1.02–1.21) per 10-point increment in FES). Greater patient-perceived UI severity (IQOL score) was not associated with better quality of UI care.
Although older persons with greater patient-perceived falls severity receive modestly better quality of care, those with more distressing incontinence do not. For both conditions, however, even the most symptomatic patients received less than half of recommended care. Low patient-perceived severity of condition is not the basis of poor care for falls and UI.
PMCID: PMC3654866  PMID: 21079526
Quality of care; Urinary Incontinence; Falls
13.  Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations 
Clinical practice guidelines are one of the foundations of efforts to improve healthcare. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearinghouses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this second paper, we discuss issues of identifying and synthesizing evidence: deciding what type of evidence and outcomes to include in guidelines; integrating values into a guideline; incorporating economic considerations; synthesis, grading, and presentation of evidence; and moving from evidence to recommendations.
PMCID: PMC3436711  PMID: 22762158
14.  Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development 
Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this third paper we discuss the issues of: reviewing, reporting, and publishing guidelines; updating guidelines; and the two emerging issues of enhancing guideline implementability and how guideline developers should approach dealing with the issue of patients who will be the subject of guidelines having co-morbid conditions.
PMCID: PMC3503794  PMID: 22762242
15.  Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest 
Clinical practice guidelines are one of the foundations of efforts to improve health care. In 1999, we authored a paper about methods to develop guidelines. Since it was published, the methods of guideline development have progressed both in terms of methods and necessary procedures and the context for guideline development has changed with the emergence of guideline clearing houses and large scale guideline production organisations (such as the UK National Institute for Health and Clinical Excellence). It therefore seems timely to, in a series of three articles, update and extend our earlier paper. In this first paper we discuss: the target audience(s) for guidelines and their use of guidelines; identifying topics for guidelines; guideline group composition (including consumer involvement) and the processes by which guideline groups function and the important procedural issue of managing conflicts of interest in guideline development.
PMCID: PMC3523009  PMID: 22762776
16.  Establishing a new journal for systematic review products 
Systematic Reviews  2012;1:1.
Welcome to a new age in publishing systematic reviews. We hope the launch of Systematic Reviews will resonate with a broad spectrum of readers interested in using them in a variety of ways, such as providing comprehensive and up to date evidence for patient management, informing health policy, and developing rigorous practice guidelines. Systematic reviews are increasingly popular. Our journal is committed to publishing a wide variety of well conducted and transparently reported systematic reviews and associated research. We are open access and electronic and not confined by space and so offer scope for publishing reviews in detail and providing a modern and innovative approach to publishing. We look forward to participating in the voyage with all of our readers.
PMCID: PMC3348672  PMID: 22587946
new journal; systematic reviews; open access
17.  Why prospective registration of systematic reviews makes sense 
Systematic Reviews  2012;1:7.
Prospective registration of systematic reviews promotes transparency, helps reduce potential for bias and serves to avoid unintended duplication of reviews. Registration offers advantages to many stakeholders in return for modest additional effort from the researchers registering their reviews.
PMCID: PMC3369816  PMID: 22588008
18.  Conducting Online Expert panels: a feasibility and experimental replicability study 
This paper has two goals. First, we explore the feasibility of conducting online expert panels to facilitate consensus finding among a large number of geographically distributed stakeholders. Second, we test the replicability of panel findings across four panels of different size.
We engaged 119 panelists in an iterative process to identify definitional features of Continuous Quality Improvement (CQI). We conducted four parallel online panels of different size through three one-week phases by using the RAND's ExpertLens process. In Phase I, participants rated potentially definitional CQI features. In Phase II, they discussed rating results online, using asynchronous, anonymous discussion boards. In Phase III, panelists re-rated Phase I features and reported on their experiences as participants.
66% of invited experts participated in all three phases. 62% of Phase I participants contributed to Phase II discussions and 87% of them completed Phase III. Panel disagreement, measured by the mean absolute deviation from the median (MAD-M), decreased after group feedback and discussion in 36 out of 43 judgments about CQI features. Agreement between the four panels after Phase III was fair (four-way kappa = 0.36); they agreed on the status of five out of eleven CQI features. Results of the post-completion survey suggest that participants were generally satisfied with the online process. Compared to participants in smaller panels, those in larger panels were more likely to agree that they had debated each others' view points.
It is feasible to conduct online expert panels intended to facilitate consensus finding among geographically distributed participants. The online approach may be practical for engaging large and diverse groups of stakeholders around a range of health services research topics and can help conduct multiple parallel panels to test for the reproducibility of panel conclusions.
PMCID: PMC3313865  PMID: 22196011
19.  Outcomes of Intravesical Botulinum Toxin for Idiopathic Overactive Bladder Symptoms: A Systematic Review of the Literature 
The Journal of urology  2010;183(6):2258-2264.
To systematically review the evidence for the efficacy and safety of botulinum toxin in the management of OAB.
Materials and Methods
We performed a systematic review of the literature to identify articles published between 1985 and March 2009 on intravesical botulinum toxin A (BTX) injections for the treatment of refractory idiopathic overactive bladder in both men and women. Database searched included MEDLINE, CENTRAL, and EMBASE. Data were tabulated from case series and from randomized controlled trials (RCTs). Data were pooled where appropriate.
Our literature search identified 432 titles. Twenty-three full articles were included in the final review. Three randomized placebo-controlled trials addressing the use of botulinum toxin-A were identified (99 patients total). The pooled random effects estimate of effect across all three studies was 3.88 (95% C.I. -6.15, -1.62), meaning that patients treated with BTX had 3.88 fewer incontinence episodes per day. UDI data revealed significant improvements in quality of life compared with placebo, with a standardized mean difference of -0.62 (CI -1.04, -0.21). Data from case series demonstrated significant improvements in OAB symptoms and quality of life, despite heterogeneity in methodology and case mix. However, based on the randomized controlled trials, there was a nine-fold increased risk of elevated post-void residual after BTX compared with placebo (8.55, 95% CI 3.22-22.71).
Intravesical injection of botulinum toxin resulted in improvement in medication-refractory OAB symptoms. However, the risk of elevated post-void residual and symptomatic urinary retention was significant. Several questions remain concerning the optimal administration of BTX for the OAB patient.
PMCID: PMC3152380  PMID: 20400142
20.  Identifying quality improvement intervention publications - A comparison of electronic search strategies 
The evidence base for quality improvement (QI) interventions is expanding rapidly. The diversity of the initiatives and the inconsistency in labeling these as QI interventions makes it challenging for researchers, policymakers, and QI practitioners to access the literature systematically and to identify relevant publications.
We evaluated search strategies developed for MEDLINE (Ovid) and PubMed based on free text words, Medical subject headings (MeSH), QI intervention components, continuous quality improvement (CQI) methods, and combinations of the strategies. Three sets of pertinent QI intervention publications were used for validation. Two independent expert reviewers screened publications for relevance. We compared the yield, recall rate, and precision of the search strategies for the identification of QI publications and for a subset of empirical studies on effects of QI interventions.
The search yields ranged from 2,221 to 216,167 publications. Mean recall rates for reference publications ranged from 5% to 53% for strategies with yields of 50,000 publications or fewer. The 'best case' strategy, a simple text word search with high face validity ('quality' AND 'improv*' AND 'intervention*') identified 44%, 24%, and 62% of influential intervention articles selected by Agency for Healthcare Research and Quality (AHRQ) experts, a set of exemplar articles provided by members of the Standards for Quality Improvement Reporting Excellence (SQUIRE) group, and a sample from the Cochrane Effective Practice and Organization of Care Group (EPOC) register of studies, respectively. We applied the search strategy to a PubMed search for articles published in 10 pertinent journals in a three-year period which retrieved 183 publications. Among these, 67% were deemed relevant to QI by at least one of two independent raters. Forty percent were classified as empirical studies reporting on a QI intervention.
The presented search terms and operating characteristics can be used to guide the identification of QI intervention publications. Even with extensive iterative development, we achieved only moderate recall rates of reference publications. Consensus development on QI reporting and initiatives to develop QI-relevant MeSH terms are urgently needed.
PMCID: PMC3170235  PMID: 21806808
21.  Identifying continuous quality improvement publications: what makes an improvement intervention ‘CQI’? 
BMJ quality & safety  2011;20(12):1011-1019.
The term continuous quality improvement (CQI) is often used to refer to a method for improving care, but no consensus statement exists on the definition of CQI. Evidence reviews are critical for advancing science, and depend on reliable definitions for article selection.
As a preliminary step towards improving CQI evidence reviews, this study aimed to use expert panel methods to identify key CQI definitional features and develop and test a screening instrument for reliably identifying articles with the key features. We used a previously published method to identify 106 articles meeting the general definition of a quality improvement intervention (QII) from 9427 electronically identified articles from PubMed. Two raters then applied a six-item CQI screen to the 106 articles.
Per cent agreement ranged from 55.7% to 75.5% for the six items, and reviewer-adjusted intra-class correlation ranged from 0.43 to 0.62. ‘Feedback of systematically collected data’ was the most common feature (64%), followed by being at least ‘somewhat’ adapted to local conditions (61%), feedback at meetings involving participant leaders (46%), using an iterative development process (40%), being at least ‘somewhat’ data driven (34%), and using a recognised change method (28%). All six features were present in 14.2% of QII articles.
We conclude that CQI features can be extracted from QII articles with reasonable reliability, but only a small proportion of QII articles include all features. Further consensus development is needed to support meaningful use of the term CQI for scientific communication.
PMCID: PMC3228263  PMID: 21727199
Continuous quality improvement; healthcare quality improvement; organisation; PDSA; research
22.  A Systematic Review of Health Care Efficiency Measures 
Health Services Research  2009;44(3):784-805.
To review and characterize existing health care efficiency measures in order to facilitate a common understanding about the adequacy of these methods.
Data Sources
Review of the MedLine and EconLit databases for articles published from 1990 to 2008, as well as search of the “gray” literature for additional measures developed by private organizations.
Study Design
We performed a systematic review for existing efficiency measures. We classified the efficiency measures by perspective, outputs, inputs, methods used, and reporting of scientific soundness.
Principal Findings
We identified 265 measures in the peer-reviewed literature and eight measures in the gray literature, with little overlap between the two sets of measures. Almost all of the measures did not explicitly consider the quality of care. Thus, if quality varies substantially across groups, which is likely in some cases, the measures reflect only the costs of care, not efficiency. Evidence on the measures' scientific soundness was mostly lacking: evidence on reliability or validity was reported for six measures (2.3 percent) and sensitivity analyses were reported for 67 measures (25.3 percent).
Efficiency measures have been subjected to few rigorous evaluations of reliability and validity, and methods of accounting for quality of care in efficiency measurement are not well developed at this time. Use of these measures without greater understanding of these issues is likely to engender resistance from providers and could lead to unintended consequences.
PMCID: PMC2699907  PMID: 19187184
Efficiency; provider profiling; performance measurement; systematic review
23.  Formative evaluation of the telecare fall prevention project for older veterans 
Fall prevention interventions for community-dwelling older adults have been found to reduce falls in some research studies. However, wider implementation of fall prevention activities in routine care has yielded mixed results. We implemented a theory-driven program to improve care for falls at our Veterans Affairs healthcare facility. The first project arising from this program used a nurse advice telephone line to identify patients' risk factors for falls and to triage patients to appropriate services. Here we report the formative evaluation of this project.
To evaluate the intervention we: 1) interviewed patient and employee stakeholders, 2) reviewed participating patients' electronic health record data and 3) abstracted information from meeting minutes. We describe the implementation process, including whether the project was implemented according to plan; identify barriers and facilitators to implementation; and assess the incremental benefit to the quality of health care for fall prevention received by patients in the project. We also estimate the cost of developing the pilot project.
The project underwent multiple changes over its life span, including the addition of an option to mail patients educational materials about falls. During the project's lifespan, 113 patients were considered for inclusion and 35 participated. Patient and employee interviews suggested support for the project, but revealed that transportation to medical care was a major barrier in following up on fall risks identified by nurse telephone triage. Medical record review showed that the project enhanced usual medical care with respect to home safety counseling. We discontinued the program after 18 months due to staffing limitations and competing priorities. We estimated a cost of $9194 for meeting time to develop the project.
The project appeared feasible at its outset but could not be sustained past the first cycle of evaluation due to insufficient resources and a waning of local leadership support due to competing national priorities. Future projects will need both front-level staff commitment and prolonged high-level leadership involvement to thrive.
PMCID: PMC3127979  PMID: 21605438
24.  Quality indicators for multiple sclerosis 
Determining whether persons with multiple sclerosis (MS) receive appropriate, comprehensive healthcare requires tools for measuring quality. The objective of this study was to develop quality indicators for the care of persons with MS. We used a modified version of the RAND/UCLA Appropriateness Method in a two-stage process to identify relevant MS care domains and to assess the validity of indicators within high-ranking care domains. Based on a literature review, interviews with persons with MS, and discussions with MS providers, 25 MS symptom domains and 14 general health domains of MS care were identified. A multidisciplinary panel of 15 stakeholders of MS care, including 4 persons with MS, rated these 39 domains in a two-round modified Delphi process. The research team performed an expanded literature review for 26 highly ranked domains to draft 86 MS care indicators. Through another two-round modified Delphi process, a second panel of 18 stakeholders rated these indicators using a nine-point response scale. Indicators with a median rating in the highest tertile were considered valid. Among the most highly rated MS care domains were appropriateness and timeliness of the diagnostic work-up, bladder dysfunction, cognition dysfunction, depression, disease-modifying agent usage, fatigue, integration of care, and spasticity. Of the 86 preliminary indicators, 76 were rated highly enough to meet predetermined thresholds for validity. Following a widely accepted methodology, we developed a comprehensive set of quality indicators for MS care that can be used to assess quality of care and guide the design of interventions to improve care among persons with MS.
PMCID: PMC2921149  PMID: 20562162
health services research; multiple sclerosis; outcome research; quality indicators
25.  Design of a continuous quality improvement program to prevent falls among community-dwelling older adults in an integrated healthcare system 
Implementing quality improvement programs that require behavior change on the part of health care professionals and patients has proven difficult in routine care. Significant randomized trial evidence supports creating fall prevention programs for community-dwelling older adults, but adoption in routine care has been limited. Nationally-collected data indicated that our local facility could improve its performance on fall prevention in community-dwelling older people. We sought to develop a sustainable local fall prevention program, using theory to guide program development.
We planned program development to include important stakeholders within our organization. The theory-derived plan consisted of 1) an initial leadership meeting to agree on whether creating a fall prevention program was a priority for the organization, 2) focus groups with patients and health care professionals to develop ideas for the program, 3) monthly workgroup meetings with representatives from key departments to develop a blueprint for the program, 4) a second leadership meeting to confirm that the blueprint developed by the workgroup was satisfactory, and also to solicit feedback on ideas for program refinement.
The leadership and workgroup meetings occurred as planned and led to the development of a functional program. The focus groups did not occur as planned, mainly due to the complexity of obtaining research approval for focus groups. The fall prevention program uses an existing telephonic nurse advice line to 1) place outgoing calls to patients at high fall risk, 2) assess these patients' risk factors for falls, and 3) triage these patients to the appropriate services. The workgroup continues to meet monthly to monitor the progress of the program and improve it.
A theory-driven program development process has resulted in the successful initial implementation of a fall prevention program.
PMCID: PMC2779811  PMID: 19917122

Results 1-25 (44)