Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful.
To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy).
We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions.
We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches.
PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.
information services; information storage and retrieval; Internet; Medline; physicians; primary health care
Physicians frequently search PubMed for information to guide patient care. More recently, Google Scholar has gained popularity as another freely accessible bibliographic database.
To compare the performance of searches in PubMed and Google Scholar.
We surveyed nephrologists (kidney specialists) and provided each with a unique clinical question derived from 100 renal therapy systematic reviews. Each physician provided the search terms they would type into a bibliographic database to locate evidence to answer the clinical question. We executed each of these searches in PubMed and Google Scholar and compared results for the first 40 records retrieved (equivalent to 2 default search pages in PubMed). We evaluated the recall (proportion of relevant articles found) and precision (ratio of relevant to nonrelevant articles) of the searches performed in PubMed and Google Scholar. Primary studies included in the systematic reviews served as the reference standard for relevant articles. We further documented whether relevant articles were available as free full-texts.
Compared with PubMed, the average search in Google Scholar retrieved twice as many relevant articles (PubMed: 11%; Google Scholar: 22%; P<.001). Precision was similar in both databases (PubMed: 6%; Google Scholar: 8%; P=.07). Google Scholar provided significantly greater access to free full-text publications (PubMed: 5%; Google Scholar: 14%; P<.001).
For quick clinical searches, Google Scholar returns twice as many relevant articles as PubMed and provides greater access to free full-text articles.
information dissemination/methods; information storage and retrieval; medical; library science; PubMed; Google Scholar; nephrology
To determine if the PubMed and Ovid MEDLINE clinical queries (which were developed in the publishing year 2000, for the purpose categories therapy, diagnosis, prognosis, etiology, and clinical prediction guides) perform as well when searching in current publishing years.
A gold standard database of recently published research literature was created using the McMaster health knowledge refinery (http://hiru.mcmaster.ca/hiru/HIRU_McMaster_HKR.aspx) and its continuously updated database, McMaster PLUS (http://hiru.mcmaster.ca/hiru/HIRU_McMaster_PLUS_projects.aspx). This database contains articles from over 120 clinical journals that are tagged for meeting or not meeting criteria for scientific merit and clinical relevance. The clinical queries sensitive (‘broad’) and specific (‘narrow’) search filters were tested in this gold standard database, and sensitivity and specificity were calculated and compared with those originally reported for the clinical queries.
In all cases, the sensitivity of the highly sensitive search filters and the specificity of the highly specific search filters did not differ substantively when comparing results derived in 2000 with those derived in a more current database. In addition, in all cases, the specificities for the highly sensitive search filters and the sensitivities for the highly specific search filters remained above 50% when testing them in the current database.
These results are reassuring for modern-day searchers. The clinical queries that were derived in the year 2000 perform equally well a decade later.
The PubMed and Ovid MEDLINE clinical queries have been revalidated and remain a useful public resource for searching the world's medical literature for research that is most relevant to clinical care.
Information storage and retrieval; Medline; Evidence-based medicine
Although the importance of detecting, treating, and controlling hypertension has been recognized for decades, the majority of patients with hypertension remain uncontrolled. The path from evidence to practice contains many potential barriers, but their role has not been reviewed systematically. This review aimed to synthesize and identify important barriers to hypertension control as reported by patients and healthcare providers.
Electronic databases MEDLINE, EMBASE and Global Health were searched systematically up to February 2013. Two reviewers independently selected eligible studies. Two reviewers categorized barriers based on a theoretical framework of behavior change. The theoretical framework suggests that a change in behavior requires a strong commitment to change [intention], the necessary skills and abilities to adopt the behavior [capability], and an absence of health system and support constraints.
Twenty-five qualitative studies and 44 quantitative studies met the inclusion criteria. In qualitative studies, health system barriers were most commonly discussed in studies of patients and health care providers. Quantitative studies identified disagreement with clinical recommendations as the most common barrier among health care providers. Quantitative studies of patients yielded different results: lack of knowledge was the most common barrier to hypertension awareness. Stress, anxiety and depression were most commonly reported as barriers that hindered or delayed adoption of a healthier lifestyle. In terms of hypertension treatment adherence, patients mostly reported forgetting to take their medication. Finally, priority setting barriers were most commonly reported by patients in terms of following up with their health care providers.
This review identified a wide range of barriers facing patients and health care providers pursuing hypertension control, indicating the need for targeted multi-faceted interventions. More methodologically rigorous studies that encompass the range of barriers and that include low- and middle-income countries are required in order to inform policies to improve hypertension control.
The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care.
To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence.
Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks–UpToDate, Physicians’ Information Education Resource (PIER), DynaMed, and Best Practice–were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated.
478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks’ treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ
3=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks).
In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.
databases, bibliographic; medical informatics; evidence-based medicine
information retrieval; MEDLINE
Clinical practice guidelines (CPGs) become quickly outdated and require a periodic reassessment of evidence research to maintain their validity. However, there is little research about this topic. Our project will provide evidence for some of the most pressing questions in this field: 1) what is the average time for recommendations to become out of date?; 2) what is the comparative performance of two restricted search strategies to evaluate the need to update recommendations?; and 3) what is the feasibility of a more regular monitoring and updating strategy compared to usual practice?. In this protocol we will focus on questions one and two.
The CPG Development Programme of the Spanish Ministry of Health developed 14 CPGs between 2008 and 2009. We will stratify guidelines by topic and by publication year, and include one CPG by strata.
We will develop a strategy to assess the validity of CPG recommendations, which includes a baseline survey of clinical experts, an update of the original exhaustive literature searches, the identification of key references (reference that trigger a potential recommendation update), and the assessment of the potential changes in each recommendation.
We will run two alternative search strategies to efficiently identify important new evidence: 1) PLUS search based in McMaster Premium LiteratUre Service (PLUS) database; and 2) a Restrictive Search (ReSe) based on the least number of MeSH terms and free text words needed to locate all the references of each original recommendation.
We will perform a survival analysis of recommendations using the Kaplan-Meier method and we will use the log-rank test to analyse differences between survival curves according to the topic, the purpose, the strength of recommendations and the turnover. We will retrieve key references from the exhaustive search and evaluate their presence in the PLUS and ReSe search results.
Our project, using a highly structured and transparent methodology, will provide guidance of when recommendations are likely to be at risk of being out of date. We will also assess two novel restrictive search strategies which could reduce the workload without compromising rigour when CPGs developers check for the need of updating.
Clinical practice guidelines; Diffusion of innovation; Dissemination and implementation; Evidence-based medicine; Information storage and retrieval; Knowledge translation; Methodology; Updating
Efficiently finding clinical examination studies—studies that quantify the value of symptoms and signs in the diagnosis of disease—is becoming increasingly difficult. Filters developed to retrieve studies of diagnosis from Medline lack specificity because they also retrieve large numbers of studies on the diagnostic value of imaging and laboratory tests.
The objective was to develop filters for retrieving clinical examination studies from Medline.
We developed filters in a training dataset and validated them in a testing database. We created the training database by hand searching 161 journals (n = 52,636 studies). We evaluated the recall and precision of 65 candidate single-term filters in identifying studies that reported the sensitivity and specificity of symptoms or signs in the training database. To identify best combinations of these search terms, we used recursive partitioning. The best-performing filters in the training database as well as 13 previously developed filters were evaluated in a testing database (n = 431,120 studies). We also examined the impact of examining reference lists of included articles on recall.
In the training database, the single-term filters with the highest recall (95%) and the highest precision (8.4%) were diagnosis[subheading] and “medical history taking”[MeSH], respectively. The multiple-term filter developed using recursive partitioning (the RP filter) had a recall of 100% and a precision of 89% in the training database. In the testing database, the Haynes-2004-Sensitive filter (recall 98%, precision 0.13%) and the RP filter (recall 89%, precision 0.52%) showed the best performance. The recall of these two filters increased to 99% and 94% respectively with review of the reference lists of the included articles.
Recursive partitioning appears to be a useful method of developing search filters. The empirical search filters proposed here can assist in the retrieval of clinical examination studies from Medline; however, because of the low precision of the search strategies, retrieving relevant studies remains challenging. Improving precision may require systematic changes in the tagging of articles by the National Library of Medicine.
Medline; filter; hedge; clinical examination; recursive partitioning
Venous thromboembolism (VTE) is a common preventable cause of mortality in hospitalized medical patients. Despite rigorous randomized trials generating strong recommendations for anticoagulant use to prevent VTE, nearly 40% of medical patients receive inappropriate thromboprophylaxis. Knowledge-translation strategies are needed to bridge this gap.
We conducted a 16-week pilot cluster randomized controlled trial (RCT) to determine the proportion of medical patients that were appropriately managed for thromboprophylaxis (according to the American College of Chest Physician guidelines) within 24 hours of admission, through the use of a multicomponent knowledge-translation intervention. Our primary goal was to determine the feasibility of conducting this study on a larger scale. The intervention comprised clinician education, a paper-based VTE risk assessment algorithm, printed physicians’ orders, and audit and feedback sessions. Medical wards at six hospitals (representing clusters) in Ontario, Canada were included; three were randomized to the multicomponent intervention and three to usual care (i.e., no active strategies for thromboprophylaxis in place). Blinding was not used.
A total of 2,611 patients (1,154 in the intervention and 1,457 in the control group) were eligible and included in the analysis. This multicomponent intervention did not lead to a significant difference in appropriate VTE prophylaxis rates between intervention and control hospitals (appropriate management rate odds ratio = 0.80; 95% confidence interval: 0.50, 1.28; p = 0.36; intra-class correlation coefficient: 0.022), and thus was not considered feasible. Major barriers to effective knowledge translation were poor attendance by clinical staff at education and feedback sessions, difficulty locating preprinted orders, and lack of involvement by clinical and administrative leaders. We identified several factors that may increase uptake of a VTE prophylaxis strategy, including local champions, support from clinical and administrative leaders, mandatory use, and a simple, clinically relevant risk assessment tool.
Hospitals allocated to our multicomponent intervention did not have a higher rate of medical inpatients appropriately managed for thromboprophylaxis than did hospitals that were not allocated to this strategy.
Thromboprophylaxis; Medical patients; Anticoagulants; Venous thromboembolism; Cluster randomization; Standard orders
Scientific knowledge is in constant change. The flow of new information requires a frequent re-evaluation of the available research results. Clinical practice guidelines (CPGs) are not exempted from this phenomenon and need to be kept updated to maintain the validity of their recommendations. The objective of our review is to systematically identify, describe and assess strategies for monitoring and updating CPGs.
Study design and setting
We conducted a systematic review of studies evaluating one or more methods of updating (with or without monitoring) CPGs or recommendations. We searched MEDLINE (PubMed) and The Cochrane Methodology Register (The Cochrane Library) from 1966 to June 2012. Additionally, we hand-searched reference lists of the included studies and the Guidelines International Network book of abstracts. If necessary, we contacted study authors to obtain additional information.
We included a total of eight studies. Four evaluated if CPGs were out of date, three updated CPGs, and one continuously monitored and updated CPGs. The most detailed reported phase of the process was the identification of new evidence. As opposed to studies updating guidelines, studies evaluating if CPGs were out of date applied restricted searches. Only one study compared a restricted versus an exhaustive search suggesting that a restricted search is sufficient to assess recommendations’ Validity. One study analyzed the survival time of CPGs and suggested that these should be reassessed every three years.
There is limited evidence about the optimal strategies for monitoring and updating clinical practice guidelines. A restricted search is likely to be sufficient to monitor new evidence and assess the need to update, however, more information is needed about the timing and type of search. Only the exhaustive search strategy has been assessed for the update of CPGs. The development and evaluation of more efficient strategies is needed to improve the timeliness and reduce the burden of maintaining the validity of CPGs.
Clinical practice guidelines; Diffusion of innovation; Evidence-based medicine; Information storage and retrieval; Methodology; Updating; Implementation science; Dissemination and implementation; Knowledge translation
When searching for renal literature, nephrologists must choose between several different bibliographic databases. We compared the availability of renal clinical studies in six major bibliographic databases.
We gathered 151 renal systematic reviews, which collectively contained 2195 unique citations referencing primary studies in the form of journal articles, meeting articles or meeting abstracts published between 1963 and 2008. We searched for each citation in three subscription-free bibliographic databases (PubMed, Google Scholar and Scirus) and three subscription-based databases (EMBASE, Ovid-MEDLINE and ISI Web of Knowledge). For the subscription-free databases, we determined which full-text journal articles were available free of charge via links to the article source.
The proportion of journal articles contained within each of the six databases ranged from 96 to 97%; results were similar for meeting articles. Availability of meeting abstracts was poor, ranging from 0 to 37% (P < 0.01) with ISI Web of Knowledge containing the largest proportion [37%, 95% confidence interval (95% CI) 32–43%]. Among the subscription-free databases, free access to full-text articles was highest in Google Scholar (38% free, 95% CI 36–41%), and was only marginally higher (39%) when all subscription-free databases were searched. After 2000, free access to full-text articles increased to 49%.
Over 99% of renal clinical journal articles are available in at least one major bibliographic database. Subscription-free databases provide free full-text access to almost half of the articles published after the year 2000, which may be of particular interest to clinicians in settings with limited access to subscription-based resources.
bibliographic databases; content coverage; evidence-based medicine; information storage and retrieval; literature searching; renal informatics
Studies published in general and specialty medical journals have the potential to improve emergency medicine (EM) practice, but there can be delayed awareness of this evidence because emergency physicians (EPs) are unlikely to read most of these journals. Also, not all published studies are intended for or ready for clinical practice application. The authors developed “Best Evidence in Emergency Medicine” (BEEM) to ameliorate these problems by searching for, identifying, appraising, and translating potentially practice-changing studies for EPs. An initial step in the BEEM process is the BEEM rater scale, a novel tool for EPs to collectively evaluate the relative clinical relevance of EM-related studies found in more than 120 journals. The BEEM rater process was designed to serve as a clinical relevance filter to identify those studies with the greatest potential to affect EM practice. Therefore, only those studies identified by BEEM raters as having the highest clinical relevance are selected for the subsequent critical appraisal process and, if found methodologically sound, are promoted as the best evidence in EM.
The primary objective was to measure inter-rater reliability (IRR) of the BEEM rater scale. Secondary objectives were to determine the minimum number of EP raters needed for the BEEM rater scale to achieve acceptable reliability and to compare performance of the scale against a previously published evidence rating system, the McMaster Online Rating of Evidence (MORE), in an EP population.
The authors electronically distributed the title, conclusion, and a PubMed link for 23 recently published studies related to EM to a volunteer group of 134 EPs. The volunteers answered two demographic questions and rated the articles using one of two randomly assigned seven-point Likert scales, the BEEM rater scale (n = 68) or the MORE scale (n = 66), over two separate administrations. The IRR of each scale was measured using generalizability theory.
The IRR of the BEEM rater scale ranged between 0.90 (95% confidence interval [CI] = 0.86 to 0.93) to 0.92 (95% CI = 0.89 to 0.94) across administrations. Decision studies showed a minimum of 12 raters is required for acceptable reliability of the BEEM rater scale. The IRR of the MORE scale was 0.82 to 0.84.
The BEEM rater scale is a highly reliable, single-question tool for a small number of EPs to collectively rate the relative clinical relevance within the specialty of EM of recently published studies from a variety of medical journals. It compares favorably with the MORE system because it achieves a high IRR despite simply requiring raters to read each article’s title and conclusion.
This study evaluated search strategies for finding high-quality studies on treatment and systematic reviews in PsycINFO.
Study design and setting:
64 journals were hand searched at McMaster University. Methodologic criteria were applied to clinically relevant articles to identify “pass” and “fail” articles. 4,985 candidate terms were compiled: 7,463 combinations for therapy articles and 5,246 combinations for reviews. Candidate search strategy results were compared with hand searches. The proposed strategies served as “diagnostic tests” for sound studies; the hand searches were the “gold standard.” Sensitivity, specificity, precision, and accuracy were calculated.
233 (32.5%) of 716 treatment articles met criteria for scientific merit, and 58 (11.5%) of 506 review articles met criteria for systematic reviews. For treatment studies, combined terms had a peak sensitivity of 97.9% (specificity 52.2%). Maximum specificity was 97.7% (sensitivity 51.5%). Sensitivity and specificity were each 79% when optimizing both while minimizing their difference. For review articles, combined terms had a peak sensitivity of 81.0% (specificity 54.4%). Maximum specificity was 98.1% (sensitivity 51.7%). Sensitivity and specificity were each 65% when optimizing both while minimizing their difference.
Empirically derived search strategies can achieve high sensitivity and specificity for retrieving sound treatment studies and review articles from PsycINFO.
Databases; bibliographic; Psychological literature; Information retrieval
Clinical Queries filters were developed to improve the retrieval of high-quality studies in searches on clinical matters. The study objective was to determine the yield of relevant citations and physician satisfaction while searching for diagnostic and treatment studies using the Clinical Queries page of PubMed compared with searching PubMed without these filters.
Materials and methods
Forty practicing physicians, presented with standardized treatment and diagnosis questions and one question of their choosing, entered search terms which were processed in a random, blinded fashion through PubMed alone and PubMed Clinical Queries. Participants rated search retrievals for applicability to the question at hand and satisfaction.
For treatment, the primary outcome of retrieval of relevant articles was not significantly different between the groups, but a higher proportion of articles from the Clinical Queries searches met methodologic criteria (p=0.049), and more articles were published in core internal medicine journals (p=0.056). For diagnosis, the filtered results returned more relevant articles (p=0.031) and fewer irrelevant articles (overall retrieval less, p=0.023); participants needed to screen fewer articles before arriving at the first relevant citation (p<0.05). Relevance was also influenced by content terms used by participants in searching. Participants varied greatly in their search performance.
Clinical Queries filtered searches returned more high-quality studies, though the retrieval of relevant articles was only statistically different between the groups for diagnosis questions.
Retrieving clinically important research studies from Medline is a challenging task for physicians. Methodological search filters can improve search retrieval.
Health information science; knowledge translation; information storage and retrieval; PubMed, search engine; databases as topic; medical informatic; Health; evidence-based medicine; information retrieval; informatics education; library science
Physicians have difficulty keeping up with new evidence from medical research.
We developed the McMaster Premium LiteratUre Service (PLUS), an internet-based addition to an existing digital library, which delivered quality- and relevance-rated medical literature to physicians, matched to their clinical disciplines. We evaluated PLUS in a cluster-randomized trial of 203 participating physicians in Northern Ontario, comparing a Full-Serve version (that included alerts to new articles and a cumulative database of alerts) with a Self-Serve version (that included a passive guide to evidence-based literature). Utilization of the service was the primary trial end-point.
Mean logins to the library rose by 0.77 logins/month/user (95% CI 0.43, 1.11) in the Full-Serve group compared with the Self-Serve group. The proportion of Full-Serve participants who utilized the service during each month of the study period showed a sustained increase during the intervention period, with a relative increase of 57% (95% CI 12, 123) compared with the Self-Serve group. There were no differences in these proportions during the baseline period, and following the crossover of the Self-Serve group to Full-Serve, the Self-Serve group’s usage became indistinguishable from that of the Full-Serve group (relative difference 4.4 (95% CI −23.7, 43.0). Also during the intervention and crossover periods, measures of self-reported usefulness did not show a difference between the 2 groups.
A quality- and relevance-rated online literature service increased the utilization of evidence-based information from a digital library by practicing physicians.
Decision Boxes are summaries of the most important benefits and harms of health interventions provided to clinicians before they meet the patient, to prepare them to help patients make informed and value-based decisions. Our objective is to explore the barriers and facilitators to using Decision Boxes in clinical practice, more precisely factors stemming from (1) the Decision Boxes themselves, (2) the primary healthcare team (PHT), and (3) the primary care practice environment.
A two-phase mixed methods study will be conducted. Eight Decision Boxes relevant to primary care, and written in both English and in French, will be hosted on a website together with a tutorial to introduce the Decision Box. The Decision Boxes will be delivered as weekly emails over a span of eight weeks to clinicians of PHTs (family physicians, residents and nurses) in five primary care clinics located across two Canadian provinces. Using a web-questionnaire, clinicians will rate each Decision Box with the Information Assessment Method (cognitive impacts, relevance, usefulness, expected benefits) and with a questionnaire based on the Theory of Planned Behavior to study the determinants of clinicians’ intention to use what they learned from that Decision Box in their patient encounter (attitude, social norm, perceived behavioral control). Web-log data will be used to monitor clinicians’ access to the website. Following the 8-week intervention, we will conduct semi-structured group interviews with clinicians and individual interviews with clinic administrators to explore contextual factors influencing the use of the Decision Boxes. Data collected from questionnaires, focus groups and individual interviews will be combined to identify factors potentially influencing implementation of Decision Boxes in clinical practice by clinicians of PHTs.
This project will allow tailoring of Decision Boxes and their delivery to overcome the specific barriers identified by clinicians of PHTs to improve the implementation of shared decision making in this setting.
(3–10); Evidence-based practice; Continuing professional education; Risk communication; Patient-centered care; Counselling; Clinical topic summary; Decision support; Knowledge translation; Implementation science
This project engages patients and physicians in the development of Decision Boxes, short clinical topic summaries covering medical questions that have no single best answer. Decision Boxes aim to prepare the clinician to communicate the risks and benefits of the available options to the patient so they can make an informed decision together.
Seven researchers (including four practicing family physicians) selected 10 clinical topics relevant to primary care practice through a Delphi survey. We then developed two one-page prototypes on two of these topics: prostate cancer screening with the prostate-specific antigen test, and prenatal screening for trisomy 21 with the serum integrated test. We presented the prototypes to purposeful samples of family physicians distributed in two focus groups, and patients distributed in four focus groups. We used the User Experience Honeycomb to explore barriers and facilitators to the communication design used in Decision Boxes. All discussions were transcribed, and three researchers proceeded to thematic content analysis of the transcriptions. The coding scheme was first developed from the Honeycomb’s seven themes (valuable, usable, credible, useful, desirable, accessible, and findable), and included new themes suggested by the data. Prototypes were modified in light of our findings.
Three rounds were necessary for a majority of researchers to select 10 clinical topics. Fifteen physicians and 33 patients participated in the focus groups. Following analyses, three sections were added to the Decision Boxes: introduction, patient counseling, and references. The information was spread to two pages to try to make the Decision Boxes less busy and improve users’ first impression. To try to improve credibility, we gave more visibility to the research institutions involved in development. A statement on the boxes’ purpose and a flow chart representing the shared decision-making process were added with the intent of clarifying the tool’s purpose. Information about the risks and benefits according to risk levels was added to the Decision Boxes, to try to ease the adaptation of the information to individual patients.
Results will guide the development of the eight remaining Decision Boxes. A future study will evaluate the effect of Decision Boxes on the integration of evidence-based and shared decision making principles in clinical practice.
Evidence-based medicine; User experience; Risk communication; Usability; Patient-centered care; Counselling; Clinical topic summary; Decision support; Knowledge translation; Communication design
It is unknown whether computer-generated, patient-tailored feedback leads to improvements in glycemic control in people with type 2 diabetes.
RESEARCH DESIGN AND METHODS
We recruited people with type 2 diabetes aged ≥40 years with a glycated hemoglobin (A1C) ≥7%, living in Hamilton, Canada, who were enrolled in a community-based program (Diabetes Hamilton) that provided regular evidence-based information and listings of community resources designed to facilitate diabetes self-management. After completing a questionnaire, participants were randomly allocated to either receive or not receive periodic computer-generated, evidence-based feedback on the basis of their questionnaire responses and designed to facilitate improved glycemic control and diabetes self-management. The primary outcome was a change in A1C after 1 year.
A total of 465 participants (50% women, mean age 62 years, and mean A1C 7.83%) were randomly assigned, and 12-month A1C values were available in 96% of all participants, at which time the A1C level had decreased by an absolute amount of 0.24 and 0.15% in the intervention and control groups, respectively. The difference in A1C reduction for the intervention versus control group was 0.09% (95% CI −0.08 to 0.26; P = 0.3). No between-group differences in measures of quality of life, diabetes self-management behaviors, or clinical outcomes were observed.
Providing computer-generated tailored feedback to registrants of a generic, community-based program that supports diabetes self-management does not lead to lower A1C levels or a better quality of life than participation in the community-based program (augmented by periodic A1C testing) alone.
Physicians practicing in ambulatory care are adopting electronic health record (EHR) systems. Governments promote this adoption with financial incentives, some hinged on improvements in care. These systems can improve care but most demonstrations of successful systems come from a few highly computerized academic environments. Those findings may not be generalizable to typical ambulatory settings, where evidence of success is largely anecdotal, with little or no use of rigorous methods. The purpose of our pilot study was to evaluate the impact of a diabetes specific chronic disease management system (CDMS) on recording of information pertinent to guideline-concordant diabetes care and to plan for larger, more conclusive studies.
Using a before–after study design we analyzed the medical record of approximately 10 patients from each of 3 diabetes specialists (total = 31) who were seen both before and after the implementation of a CDMS. We used a checklist of key clinical data to compare the completeness of information recorded in the CDMS record to both the clinical note sent to the primary care physician based on that same encounter and the clinical note sent to the primary care physician based on the visit that occurred prior to the implementation of the CDMS, accounting for provider effects with Generalized Estimating Equations.
The CDMS record outperformed by a substantial margin dictated notes created for the same encounter. Only 10.1% (95% CI, 7.7% to 12.3%) of the clinically important data were missing from the CDMS chart compared to 25.8% (95% CI, 20.5% to 31.1%) from the clinical note prepared at the time (p < 0.001) and 26.3% (95% CI, 19.5% to 33.0%) from the clinical note prepared before the CDMS was implemented (p < 0.001). There was no significant difference between dictated notes created for the CDMS-assisted encounter and those created for usual care encounters (absolute mean difference, 0.8%; 95% CI, −8.5% to 6.8%).
The CDMS chart captured information important for the management of diabetes more often than dictated notes created with or without its use but we were unable to detect a difference in completeness between notes dictated in CDMS-associated and usual-care encounters. Our sample of patients and providers was small, and completeness of records may not reflect quality of care.
Background: Clinical end users of EMBASE have a difficult time retrieving articles that are both scientifically sound and directly relevant to clinical practice. Search filters have been developed to assist end users in increasing the success of their searches. Many filters have been developed for the literature on therapy and reviews for use in MEDLINE, but little has been done for use in EMBASE with no filter development for studies of prognosis. The objective of this study was to determine how well various methodologic textwords, index terms, and their Boolean combinations retrieve methodologically sound literature on the prognosis of health disorders in EMBASE.
Methods: An analytic survey was conducted, comparing hand searches of 55 journals with retrievals from EMBASE for 4,843 candidate search terms and 8,919 combinations. All articles were rated using purpose and quality indicators, and clinically relevant prognostic articles were categorized as “pass” or “fail” according to explicit criteria for scientific merit. Candidate search strategies were run in EMBASE, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated.
Results: Of the 1,064 articles about prognosis, 148 (13.9%) met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 98.7% with specificity at 50.6%. Compared with best single terms, best multiple terms increased sensitivity for sound studies by 12.2% (absolute increase), while decreasing specificity (absolute decrease 5.1%) when sensitivity was maximized. Combinations of search terms reached peak specificities of 93.4% with sensitivity at 50.7%. Compared with best single terms, best multiple terms increased specificity for sound studies by 7.1% (absolute increase), while decreasing sensitivity (absolute decrease 8.8%) when specificity was maximized.
Conclusion: Empirically derived search strategies combining indexing terms and textwords can achieve high sensitivity or specificity for retrieving sound prognostic studies from EMBASE.
Tools to enhance physician searches of Medline and other bibliographic databases have potential to improve the application of new knowledge in patient care. This is particularly true for articles about glomerular disease, which are published across multiple disciplines and are often difficult to track down. Our objective was to develop and test search filters for PubMed, Ovid Medline, and Embase that allow physicians to search within a subset of the database to retrieve articles relevant to glomerular disease.
We used a diagnostic test assessment framework with development and validation phases. We read a total of 22,992 full text articles for relevance and assigned them to the development or validation set to define the reference standard. We then used combinations of search terms to develop 997,298 unique glomerular disease filters. Outcome measures for each filter included sensitivity, specificity, precision, and accuracy. We selected optimal sensitive and specific search filters for each database and applied them to the validation set to test performance.
High performance filters achieved at least 93.8% sensitivity and specificity in the development set. Filters optimized for sensitivity reached at least 96.7% sensitivity and filters optimized for specificity reached at least 98.4% specificity. Performance of these filters was consistent in the validation set and similar among all three databases.
PubMed, Ovid Medline, and Embase can be filtered for articles relevant to glomerular disease in a reliable manner. These filters can now be used to facilitate physician searching.
Glomerular diseases; Glomerulopathy; Medical Informatics; Information retrieval; Medline; Embase
The long-term health consequences of acute bacterial gastroenteritis remain uncertain. We studied the risk of hypertension and reduced kidney function after an outbreak of acute gastroenteritis due to contamination of a regional drinking water supply with Escherichia coli O157:H7 and Campylobacter species.
A total of 1958 adults with no known history of hypertension or kidney disease before the outbreak participated in a long-term follow-up study. Of the participants, 675 had been asymptomatic during the outbreak, 909 had had moderate symptoms of acute self-limited gastroenteritis, and 374 had had severe symptoms that necessitated medical attention. The outcomes of interest were a diagnosis of hypertension or the presence of reduced kidney function and albuminuria during the follow-up period.
After a mean follow-up of 3.7 years after the outbreak, hypertension was diagnosed in 27.0% of participants who had been asymptomatic during the outbreak and in 32.3% and 35.9% of those who had had moderate and severe symptoms of acute gastroenteritis respectively (trend p = 0.009). Compared with the asymptomatic participants, those with moderate and severe symptoms of gastroenteritis had an adjusted relative risk of hypertension of 1.15 (95% confidence interval [CI] 0.97–1.35) and 1.28 (95% CI 1.04–1.56) respectively. A similar graded association was seen for reduced kidney function, defined as the presence of an estimated glomerular filtration rate below 60 mL/min per 1.73 m2 (trend p = 0.03). No association was observed between gastroenteritis and the subsequent risk of albuminuria.
Acute bacterial gastroenteritis necessitating medical attention was associated with an increased risk of hypertension and reduced kidney function 4 years after infection. Maintaining safe drinking water remains essential to human health, as transient bacterial contaminations may have implications well beyond a period of acute self-limited illness.
Evidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards.
The retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25 936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies.
A majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%.
Sensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.
Physicians face challenges when searching PubMed for research evidence, and they may miss relevant articles while retrieving too many nonrelevant articles. We investigated whether the use of search filters in PubMed improves searching by physicians.
We asked a random sample of Canadian nephrologists to answer unique clinical questions derived from 100 systematic reviews of renal therapy. Physicians provided the search terms that they would type into PubMed to locate articles to answer these questions. We entered the physician-provided search terms into PubMed and applied two types of search filters alone or in combination: a methods-based filter designed to identify high-quality studies about treatment (clinical queries “therapy”) and a topic-based filter designed to identify studies with renal content. We evaluated the comprehensiveness (proportion of relevant articles found) and efficiency (ratio of relevant to nonrelevant articles) of the filtered and nonfiltered searches. Primary studies included in the systematic reviews served as the reference standard for relevant articles.
The average physician-provided search terms retrieved 46% of the relevant articles, while 6% of the retrieved articles were nonrelevant (the ratio of relevant to nonrelevant articles was 1:16). The use of both filters together produced a marked improvement in efficiency, resulting in a ratio of relevant to nonrelevant articles of 1:5 (16 percentage point improvement; 99% confidence interval 9% to 22%; p < 0.003) with no substantive change in comprehensiveness (44% of relevant articles found; p = 0.55).
The use of PubMed search filters improves the efficiency of physician searches. Improved search performance may enhance the transfer of research into practice and improve patient care.