The research attempted to develop search filters for biomedical literature databases that improve retrieval of studies of clinical relevance for the nursing and rehabilitation professions.
Diagnostic testing framework compared machine-culled and practitioner-nominated search terms with a hand-tagged clinical literature database.
We were unable to: (1) develop filters for nursing, likely because of the overlapping and expanding scope of practice for nurses in comparison with medical professionals, or (2) develop filters for rehabilitation, because of its broad scope and the profession's multifaceted understanding of “health and ability.”
We found limitations on search filter development for these health professions: nursing and rehabilitation.
Nursing; Occupational Therapy; Physical Therapy Specialty; Databases, Bibliographic; Search Engine; Terminology as Topic; Information Storage and Retrieval
We developed an evidence service that draws inputs from Health Systems Evidence (HSE), which is a comprehensive database of research evidence about governance, financial and delivery arrangements within health systems and about implementation strategies relevant to health systems. Our goal was to evaluate whether, how and why a ‘full-serve’ evidence service increases the use of synthesized research evidence by policy analysts and advisors in the Ontario Ministry of Health and Long-Term Care as compared to a ‘self-serve’ evidence service.
We attempted to conduct a two-arm, 10-month randomized controlled trial (RCT), along with a follow-up qualitative process evaluation, but we terminated the RCT when we failed to reach our recruitment target. For the qualitative process evaluation we modified the original interview guide to allow us to explore the (1) factors influencing participation in the trial; (2) usage of HSE, factors explaining usage patterns, and strategies to increase usage; (3) participation in training workshops and use of other supports; and (4) views about and experiences with key HSE features.
We terminated the RCT given our 15% recruitment rate. Six factors were identified by those who had agreed to participate in the trial as encouraging their participation: relevance of the study to participants’ own work; familiarity with the researchers; personal view of the importance of using research evidence in policymaking; academic background; support from supervisors; and participation of colleagues. Most reported that they never, infrequently or inconsistently used HSE and suggested strategies to increase its use, including regular email reminders and employee training. However, only two participants indicated that employee training, in the form of a workshop about finding and using research evidence, had influenced their use of HSE. Most participants found HSE features to be intuitive and helpful, although registration/sign-in and some page formats (particularly the advanced search page and detailed search results page) discouraged their use or did not optimize the user experience.
The qualitative findings informed a re-design of HSE, which allows users to more efficiently find and use research evidence about how to strengthen or reform health systems or in how to get cost-effective programs, services and drugs to those who need them. Our experience with RCT recruitment suggests the need to consider changing the unit of allocation to divisions instead of individuals within divisions, among other lessons.
This protocol for this study is published in Implementation Science and registered with ClinicalTrials.gov (HHS/FHS REB 10–267).
Electronic supplementary material
The online version of this article (doi:10.1186/s12961-015-0066-z) contains supplementary material, which is available to authorized users.
Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful.
To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy).
We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions.
We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches.
PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.
information services; information storage and retrieval; Internet; Medline; physicians; primary health care
The article “Users’ guide to the surgical literature: how to perform a literature search” was published in 2003, but the continuing technological developments in databases and search filters have rendered that guide out of date. The present guide fills an existing gap in this area; it provides the reader with strategies for developing a searchable clinical question, creating an efficient search strategy, accessing appropriate databases, and skillfully retrieving the best evidence to address the research question.
Physicians frequently search PubMed for information to guide patient care. More recently, Google Scholar has gained popularity as another freely accessible bibliographic database.
To compare the performance of searches in PubMed and Google Scholar.
We surveyed nephrologists (kidney specialists) and provided each with a unique clinical question derived from 100 renal therapy systematic reviews. Each physician provided the search terms they would type into a bibliographic database to locate evidence to answer the clinical question. We executed each of these searches in PubMed and Google Scholar and compared results for the first 40 records retrieved (equivalent to 2 default search pages in PubMed). We evaluated the recall (proportion of relevant articles found) and precision (ratio of relevant to nonrelevant articles) of the searches performed in PubMed and Google Scholar. Primary studies included in the systematic reviews served as the reference standard for relevant articles. We further documented whether relevant articles were available as free full-texts.
Compared with PubMed, the average search in Google Scholar retrieved twice as many relevant articles (PubMed: 11%; Google Scholar: 22%; P<.001). Precision was similar in both databases (PubMed: 6%; Google Scholar: 8%; P=.07). Google Scholar provided significantly greater access to free full-text publications (PubMed: 5%; Google Scholar: 14%; P<.001).
For quick clinical searches, Google Scholar returns twice as many relevant articles as PubMed and provides greater access to free full-text articles.
information dissemination/methods; information storage and retrieval; medical; library science; PubMed; Google Scholar; nephrology
The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care.
To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence.
Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks–UpToDate, Physicians’ Information Education Resource (PIER), DynaMed, and Best Practice–were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated.
478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks’ treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ
3=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks).
In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.
databases, bibliographic; medical informatics; evidence-based medicine
information retrieval; MEDLINE
Heart failure (HF) is the commonest cause of hospitalization in older adults. Compared to routine hospitalization (RH), hospital at home (HaH)—substitutive hospital-level care in the patient’s home—improves outcomes and reduces costs in patients with general medical conditions. The efficacy of HaH in HF is unknown.
Methods and Results
We searched MEDLINE, Embase, CINAHL, and CENTRAL, for publications from January 1990 to October 2014. We included prospective studies comparing substitutive models of hospitalization to RH in HF. At least 2 reviewers independently selected studies, abstracted data, and assessed quality. We meta-analyzed results from 3 RCTs (n = 203) and narratively synthesized results from 3 observational studies (n = 329). Study quality was modest. In RCTs, HaH increased time to first readmission (mean difference (MD) 14.13 days [95% CI 10.36 to 17.91]), and improved health-related quality of life (HrQOL) at both, 6 months (standardized MD (SMD) -0.31 [-0.45 to -0.18]) and 12 months (SMD -0.17 [-0.31 to -0.02]). In RCTs, HaH demonstrated a trend to decreased readmissions (risk ratio (RR) 0.68 [0.42 to 1.09]), and had no effect on all-cause mortality (RR 0.94 [0.67 to 1.32]). HaH decreased costs of index hospitalization in all RCTs. HaH reduced readmissions and emergency department visits per patient in all 3 observational studies.
In the context of a limited number of modest-quality studies, HaH appears to increase time to readmission, reduce index costs, and improve HrQOL among patients requiring hospital-level care for HF. Larger RCTs are necessary to assess the effect of HaH on readmissions, mortality, and long-term costs.
Efficiently finding clinical examination studies—studies that quantify the value of symptoms and signs in the diagnosis of disease—is becoming increasingly difficult. Filters developed to retrieve studies of diagnosis from Medline lack specificity because they also retrieve large numbers of studies on the diagnostic value of imaging and laboratory tests.
The objective was to develop filters for retrieving clinical examination studies from Medline.
We developed filters in a training dataset and validated them in a testing database. We created the training database by hand searching 161 journals (n = 52,636 studies). We evaluated the recall and precision of 65 candidate single-term filters in identifying studies that reported the sensitivity and specificity of symptoms or signs in the training database. To identify best combinations of these search terms, we used recursive partitioning. The best-performing filters in the training database as well as 13 previously developed filters were evaluated in a testing database (n = 431,120 studies). We also examined the impact of examining reference lists of included articles on recall.
In the training database, the single-term filters with the highest recall (95%) and the highest precision (8.4%) were diagnosis[subheading] and “medical history taking”[MeSH], respectively. The multiple-term filter developed using recursive partitioning (the RP filter) had a recall of 100% and a precision of 89% in the training database. In the testing database, the Haynes-2004-Sensitive filter (recall 98%, precision 0.13%) and the RP filter (recall 89%, precision 0.52%) showed the best performance. The recall of these two filters increased to 99% and 94% respectively with review of the reference lists of the included articles.
Recursive partitioning appears to be a useful method of developing search filters. The empirical search filters proposed here can assist in the retrieval of clinical examination studies from Medline; however, because of the low precision of the search strategies, retrieving relevant studies remains challenging. Improving precision may require systematic changes in the tagging of articles by the National Library of Medicine.
Medline; filter; hedge; clinical examination; recursive partitioning
Decision boxes (DBoxes) are two-page evidence summaries to prepare clinicians for shared decision making (SDM). We sought to assess the feasibility of a clustered Randomized Controlled Trial (RCT) to evaluate their impact.
A convenience sample of clinicians (nurses, physicians and residents) from six primary healthcare clinics who received eight DBoxes and rated their interest in the topic and satisfaction. After consultations, their patients rated their involvement in decision-making processes (SDM-Q-9 instrument). We measured clinic and clinician recruitment rates, questionnaire completion rates, patient eligibility rates, and estimated the RCT needed sample size.
Among the 20 family medicine clinics invited to participate in this study, four agreed to participate, giving an overall recruitment rate of 20%. Of 148 clinicians invited to the study, 93 participated (63%). Clinicians rated an interest in the topics ranging 6.4-8.2 out of 10 (with 10 highest) and a satisfaction with DBoxes of 4 or 5 out of 5 (with 5 highest) for 81% DBoxes. For the future RCT, we estimated that a sample size of 320 patients would allow detecting a 9% mean difference in the SDM-Q-9 ratings between our two arms (0.02 ICC; 0.05 significance level; 80% power).
Clinicians’ recruitment and questionnaire completion rates support the feasibility of the planned RCT. The level of interest of participants for the DBox topics, and their level of satisfaction with the Dboxes demonstrate the acceptability of the intervention. Processes to recruit clinics and patients should be optimized.
Electronic supplementary material
The online version of this article (doi:10.1186/s12911-015-0134-x) contains supplementary material, which is available to authorized users.
Evidence-based practice; Continuing professional education; Shared decision making; Risk communication; Patient-centered care; Counseling; Clinical topic summary; Decision support; Knowledge translation
A validated and reliable instrument was developed to knowledge, attitudes and behaviours with respect to evidence-based practice (EBB-KABQ) in medical trainees but requires further adaptation and validation to be applied across different health professionals.
A modified 33-item evidence-based practice scale (EBP-KABQ) was developed to evaluate EBP perceptions and behaviors in clinicians. An international sample of 673 clinicians interested in treatment of pain (mean age = 45 years, 48% occupational therapists/physical therapists, 25% had more than 5 years of clinical training) completed an online English version of the questionnaire and demographics. Scaling properties (internal consistency, floor/ceiling effects) and construct validity (association with EBP activities, comparator constructs) were examined. A confirmatory factor analysis was used to assess the 4-domain structure EBP knowledge, attitudes, behavior, outcomes/decisions).
The EBP-KABQ scale demonstrated high internal consistency (Cronbach’s alpha = 0.85), no evident floor/ceiling effects, and support for a priori construct validation hypotheses. A 4-factor structure provided the best fit statistics (CFI =0.89, TLI =0.86, and RMSEA = 0.06).
The EBP-KABQ scale demonstrates promising psychometric properties in this sample. Areas for improvement are described.
Electronic supplementary material
The online version of this article (doi:10.1186/s12909-014-0263-4) contains supplementary material, which is available to authorized users.
Evidence-based; Scale; Self-reported; Validation; Clinician
This study evaluated search strategies for finding high-quality studies on treatment and systematic reviews in PsycINFO.
Study design and setting:
64 journals were hand searched at McMaster University. Methodologic criteria were applied to clinically relevant articles to identify “pass” and “fail” articles. 4,985 candidate terms were compiled: 7,463 combinations for therapy articles and 5,246 combinations for reviews. Candidate search strategy results were compared with hand searches. The proposed strategies served as “diagnostic tests” for sound studies; the hand searches were the “gold standard.” Sensitivity, specificity, precision, and accuracy were calculated.
233 (32.5%) of 716 treatment articles met criteria for scientific merit, and 58 (11.5%) of 506 review articles met criteria for systematic reviews. For treatment studies, combined terms had a peak sensitivity of 97.9% (specificity 52.2%). Maximum specificity was 97.7% (sensitivity 51.5%). Sensitivity and specificity were each 79% when optimizing both while minimizing their difference. For review articles, combined terms had a peak sensitivity of 81.0% (specificity 54.4%). Maximum specificity was 98.1% (sensitivity 51.7%). Sensitivity and specificity were each 65% when optimizing both while minimizing their difference.
Empirically derived search strategies can achieve high sensitivity and specificity for retrieving sound treatment studies and review articles from PsycINFO.
Databases; bibliographic; Psychological literature; Information retrieval
Clinical guidelines should be updated to maintain their validity. Our aim was to estimate the length of time before recommendations become outdated.
We used a retrospective cohort design and included recommendations from clinical guidelines developed in the Spanish National Health System clinical guideline program since 2008. We performed a descriptive analysis of references, recommendations and resources used, and a survival analysis of recommendations using the Kaplan–Meier method.
We included 113 recommendations from 4 clinical guidelines with a median of 4 years since the most recent search (range 3.9–4.4 yr). We retrieved 39 136 references (range 3343–14 787) using an exhaustive literature search, 668 of which were related to the recommendations in our sample. We identified 69 (10.3%) key references, corresponding to 25 (22.1%) recommendations that required updating. Ninety-two percent (95% confidence interval 86.9–97.0) of the recommendations were valid 1 year after their development. This probability decreased at 2 (85.7%), 3 (81.3%) and 4 years (77.8%).
Recommendations quickly become outdated, with 1 out of 5 recommendations being out of date after 3 years. Waiting more than 3 years to review a guideline is potentially too long.
Decision boxes (Dboxes) provide clinicians with research evidence about management options for medical questions that have no single best answer. Dboxes fulfil a need for rapid clinical training tools to prepare clinicians for clinician-patient communication and shared decision-making. We studied the barriers and facilitators to using the Dbox information in clinical practice.
We used a mixed methods study with sequential explanatory design. We recruited family physicians, residents, and nurses from six primary health-care clinics. Participants received eight Dboxes covering various questions by email (one per week). For each Dbox, they completed a web questionnaire to rate clinical relevance and cognitive impact and to assess the determinants of their intention to use what they learned from the Dbox to explain to their patients the advantages and disadvantages of the options, based on the theory of planned behaviour (TPB). Following the 8-week delivery period, we conducted focus groups with clinicians and interviews with clinic administrators to explore contextual factors influencing the use of the Dbox information.
One hundred clinicians completed the web surveys. In 54% of the 496 questionnaires completed, they reported that their practice would be improved after having read the Dboxes, and in 40%, they stated that they would use this information for their patients. Of those who would use the information for their patients, 89% expected it would benefit their patients, especially in that it would allow the patient to make a decision more in keeping with his/her personal circumstances, values, and preferences. They intended to use the Dboxes in practice (mean 5.6 ± 1.2, scale 1–7, with 7 being “high”), and their intention was significantly related to social norm, perceived behavioural control, and attitude according to the TPB (P < 0.0001). In focus groups, clinicians mentioned that co-interventions such as patient decision aids and training in shared decision-making would facilitate the use of the Dbox information. Some participants would have liked a clear “bottom line” statement for each Dbox and access to printed Dboxes in consultation rooms.
Dboxes are valued by clinicians. Tailoring of Dboxes to their needs would facilitate their implementation in practice.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0144-6) contains supplementary material, which is available to authorized users.
Clinical practice guidelines; Knowledge translation; Decision support; Evidence-based practice; Barriers; Patient-centred care; User experience; Continuing professional development; Communication competency
Physicians have difficulty keeping up with new evidence from medical research.
We developed the McMaster Premium LiteratUre Service (PLUS), an internet-based addition to an existing digital library, which delivered quality- and relevance-rated medical literature to physicians, matched to their clinical disciplines. We evaluated PLUS in a cluster-randomized trial of 203 participating physicians in Northern Ontario, comparing a Full-Serve version (that included alerts to new articles and a cumulative database of alerts) with a Self-Serve version (that included a passive guide to evidence-based literature). Utilization of the service was the primary trial end-point.
Mean logins to the library rose by 0.77 logins/month/user (95% CI 0.43, 1.11) in the Full-Serve group compared with the Self-Serve group. The proportion of Full-Serve participants who utilized the service during each month of the study period showed a sustained increase during the intervention period, with a relative increase of 57% (95% CI 12, 123) compared with the Self-Serve group. There were no differences in these proportions during the baseline period, and following the crossover of the Self-Serve group to Full-Serve, the Self-Serve group’s usage became indistinguishable from that of the Full-Serve group (relative difference 4.4 (95% CI −23.7, 43.0). Also during the intervention and crossover periods, measures of self-reported usefulness did not show a difference between the 2 groups.
A quality- and relevance-rated online literature service increased the utilization of evidence-based information from a digital library by practicing physicians.
Background & aims
Finding current best evidence for clinical decisions remains challenging. With 3,000 new studies published every day, no single evidence-based resource provides all answers or is sufficiently updated. McMaster Premium LiteratUre Service – Federated Search (MacPLUS FS) addresses this issue by looking in multiple high quality resources simultaneously and displaying results in a one-page pyramid with the most clinically useful at the top. Yet, additional logistical and educational barriers need to be addressed to enhance point-of-care evidence retrieval. This trial seeks to test three innovative interventions, among clinicians registered to MacPLUS FS, to increase the quantity and quality of searching for current best evidence to answer clinical questions.
Methods & design
In a user-centered approach, we designed three interventions embedded in MacPLUS FS: (A) a web-based Clinical Question Recorder; (B) an Evidence Retrieval Coach composed of eight short educational videos; (C) an Audit, Feedback and Gamification approach to evidence retrieval, based on the allocation of ‘badges’ and ‘reputation scores.’
We will conduct a randomized factorial controlled trial among all the 904 eligible medical doctors currently registered to MacPLUS FS at the hospitals affiliated with McMaster University, Canada. Postgraduate trainees (n = 429) and clinical faculty/staff (n = 475) will be randomized to each of the three following interventions in a factorial design (A x B x C). Utilization will be continuously recorded through clinicians’ accounts that track logins and usage, down to the level of individual keystrokes. The primary outcome is the rate of searches per month per user during the six months of follow-up. Secondary outcomes, measured through the validated Impact Assessment Method questionnaire, include: utility of answers found (meeting clinicians’ information needs), use (application in practice), and perceived usefulness on patient outcomes.
Built on effective models for the point-of-care teaching, these interventions approach evidence retrieval as a clinical skill. If effective, they may offer the opportunity to enhance it for a large audience, at low cost, providing better access to relevant evidence across many top EBM resources in parallel.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0125-9) contains supplementary material, which is available to authorized users.
Evidence-based medicine; Evidence retrieval; Knowledge translation; Audit and feedback; Web-based resources; Search engines
Background: Clinical end users of EMBASE have a difficult time retrieving articles that are both scientifically sound and directly relevant to clinical practice. Search filters have been developed to assist end users in increasing the success of their searches. Many filters have been developed for the literature on therapy and reviews for use in MEDLINE, but little has been done for use in EMBASE with no filter development for studies of prognosis. The objective of this study was to determine how well various methodologic textwords, index terms, and their Boolean combinations retrieve methodologically sound literature on the prognosis of health disorders in EMBASE.
Methods: An analytic survey was conducted, comparing hand searches of 55 journals with retrievals from EMBASE for 4,843 candidate search terms and 8,919 combinations. All articles were rated using purpose and quality indicators, and clinically relevant prognostic articles were categorized as “pass” or “fail” according to explicit criteria for scientific merit. Candidate search strategies were run in EMBASE, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated.
Results: Of the 1,064 articles about prognosis, 148 (13.9%) met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 98.7% with specificity at 50.6%. Compared with best single terms, best multiple terms increased sensitivity for sound studies by 12.2% (absolute increase), while decreasing specificity (absolute decrease 5.1%) when sensitivity was maximized. Combinations of search terms reached peak specificities of 93.4% with sensitivity at 50.7%. Compared with best single terms, best multiple terms increased specificity for sound studies by 7.1% (absolute increase), while decreasing sensitivity (absolute decrease 8.8%) when specificity was maximized.
Conclusion: Empirically derived search strategies combining indexing terms and textwords can achieve high sensitivity or specificity for retrieving sound prognostic studies from EMBASE.
The long-term health consequences of acute bacterial gastroenteritis remain uncertain. We studied the risk of hypertension and reduced kidney function after an outbreak of acute gastroenteritis due to contamination of a regional drinking water supply with Escherichia coli O157:H7 and Campylobacter species.
A total of 1958 adults with no known history of hypertension or kidney disease before the outbreak participated in a long-term follow-up study. Of the participants, 675 had been asymptomatic during the outbreak, 909 had had moderate symptoms of acute self-limited gastroenteritis, and 374 had had severe symptoms that necessitated medical attention. The outcomes of interest were a diagnosis of hypertension or the presence of reduced kidney function and albuminuria during the follow-up period.
After a mean follow-up of 3.7 years after the outbreak, hypertension was diagnosed in 27.0% of participants who had been asymptomatic during the outbreak and in 32.3% and 35.9% of those who had had moderate and severe symptoms of acute gastroenteritis respectively (trend p = 0.009). Compared with the asymptomatic participants, those with moderate and severe symptoms of gastroenteritis had an adjusted relative risk of hypertension of 1.15 (95% confidence interval [CI] 0.97–1.35) and 1.28 (95% CI 1.04–1.56) respectively. A similar graded association was seen for reduced kidney function, defined as the presence of an estimated glomerular filtration rate below 60 mL/min per 1.73 m2 (trend p = 0.03). No association was observed between gastroenteritis and the subsequent risk of albuminuria.
Acute bacterial gastroenteritis necessitating medical attention was associated with an increased risk of hypertension and reduced kidney function 4 years after infection. Maintaining safe drinking water remains essential to human health, as transient bacterial contaminations may have implications well beyond a period of acute self-limited illness.
Evidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards.
The retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25 936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies.
A majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%.
Sensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.
Objective: To identify a journal subset that publishes reports of high quality studies and reviews relating to age-specific clinical specialties, such as pediatrics and geriatrics.
Design: Handsearch of 172 journals using explicit criteria to determine methodologic quality for generating evidence for clinical practice.
Main outcome measure: Frequency of high quality articles and their top yielding journals.
Results: Between 17% and 33% of articles published in age-specific specialties are of high quality for clinical use. Top yielding journals for the specialties ranged from 16 to 130.
Conclusion: Handsearch of the clinical literature for the year 2000 reveals that high quality articles for some age-specific specialties are concentrated in a small subset of journals (eg, obstetrics), whereas articles for other specialties are widely scattered among a large number of journals (eg, adult medicine).
Systematic review articles (SRs) are becoming increasingly important to clinical practice. A hand search of the core health care journals was done to assess number and type of SRs for 2000. Many more SRs are being published but they are concentrated in The Cochrane Database of Systematic Reviews (CDSR) and a few core clinical journals. Their topics are also concentrated on therapy issues rather than other categories of health research. Background: SRs have assumed increasing importance for evidence-based patient care and services decisions. They summarize data from all quality-assessed studies for a health problem and provide scientific synthesis.
Objective - To determine the core journals for clinical practice in infectious disease (ID) by examination of citations in The Cochrane Database of Systematic Reviews (CDSR). Methods - References included in ID-related systematic reviews (SRs) of the CDSR were assessed to establish the journals that publish articles of high quality and clinical relevance. Results - 152 ID-related SRs cited 1374 articles published in 342 journals. 50% of the citations appeared in 28 journals and 51.5% of the journals were cited only once. The Lancet was most frequently cited, with 74 citations (5.4%). Conclusion - Studies of ID interventions are scattered over 300 journals.
To determine if the PubMed and Ovid MEDLINE clinical queries (which were developed in the publishing year 2000, for the purpose categories therapy, diagnosis, prognosis, etiology, and clinical prediction guides) perform as well when searching in current publishing years.
A gold standard database of recently published research literature was created using the McMaster health knowledge refinery (http://hiru.mcmaster.ca/hiru/HIRU_McMaster_HKR.aspx) and its continuously updated database, McMaster PLUS (http://hiru.mcmaster.ca/hiru/HIRU_McMaster_PLUS_projects.aspx). This database contains articles from over 120 clinical journals that are tagged for meeting or not meeting criteria for scientific merit and clinical relevance. The clinical queries sensitive (‘broad’) and specific (‘narrow’) search filters were tested in this gold standard database, and sensitivity and specificity were calculated and compared with those originally reported for the clinical queries.
In all cases, the sensitivity of the highly sensitive search filters and the specificity of the highly specific search filters did not differ substantively when comparing results derived in 2000 with those derived in a more current database. In addition, in all cases, the specificities for the highly sensitive search filters and the sensitivities for the highly specific search filters remained above 50% when testing them in the current database.
These results are reassuring for modern-day searchers. The clinical queries that were derived in the year 2000 perform equally well a decade later.
The PubMed and Ovid MEDLINE clinical queries have been revalidated and remain a useful public resource for searching the world's medical literature for research that is most relevant to clinical care.
Information storage and retrieval; Medline; Evidence-based medicine