Search tips
Search criteria

Results 1-25 (30)

Clipboard (0)

Select a Filter Below

Year of Publication
more »
Document Types
1.  Building better guidelines with BRIDGE-Wiz: development and evaluation of a software assistant to promote clarity, transparency, and implementability 
To demonstrate the feasibility of capturing the knowledge required to create guideline recommendations in a systematic, structured, manner using a software assistant. Practice guidelines constitute an important modality that can reduce the delivery of inappropriate care and support the introduction of new knowledge into clinical practice. However, many guideline recommendations are vague and underspecified, lack any linkage to supporting evidence or documentation of how they were developed, and prove to be difficult to transform into systems that influence the behavior of care providers.
The BRIDGE-Wiz application (Building Recommendations In a Developer's Guideline Editor) uses a wizard approach to address the questions: (1) under what circumstances? (2) who? (3) ought (with what level of obligation?) (4) to do what? (5) to whom? (6) how and why? Controlled natural language was applied to create and populate a template for recommendation statements.
The application was used by five national panels to develop guidelines. In general, panelists agreed that the software helped to formalize a process for authoring guideline recommendations and deemed the application usable and useful.
Use of BRIDGE-Wiz promotes clarity of recommendations by limiting verb choices, building active voice recommendations, incorporating decidability and executability checks, and limiting Boolean connectors. It enhances transparency by incorporating systematic appraisal of evidence quality, benefits, and harms. BRIDGE-Wiz promotes implementability by providing a pseudocode rule, suggesting deontic modals, and limiting the use of ‘consider’.
Users found that BRIDGE-Wiz facilitates the development of clear, transparent, and implementable guideline recommendations.
PMCID: PMC3240753  PMID: 21846779
Controlled natural language; controlled terminologies and vocabularies; developing/using clinical decision support (other than diagnostic) and guideline systems; developing/using wireless and in-the-field applications (mHealth); guideline development; guidelines; implementation; knowledge bases; knowledge representations; ontologies; portable; practice guidelines
2.  Evaluating the use of a computerized clinical decision support system for asthma by pediatric pulmonologists 
To investigate use of a new guideline-based, computerized clinical decision support (CCDS) system for asthma in a pediatric pulmonology clinic of a large academic medical center.
We conducted a qualitative evaluation including review of electronic data, direct observation, and interviews with all nine pediatric pulmonologists in the clinic. Outcome measures included patterns of computer use in relation to patient care, and themes surrounding the relationship between asthma care and computer use.
The pediatric pulmonologists entered enough data to trigger the decision support system in 397/445 (89.2%) of all asthma visits from January 2009 to May 2009. However, interviews and direct observations revealed use of the decision support system was limited to documentation activities after clinic sessions ended. Reasons for delayed use reflected barriers common to general medical care and barriers specific to subspecialty care. Subspecialist-specific barriers included the perceived high complexity of patients, the impact of subject matter expertise on the types of decision support needed, and unique workflow concerns such as the need to create letters to referring physicians.
Pediatric pulmonologists demonstrated low use of a computerized decision support system for asthma care because of a combination of general and subspecialist-specific factors. Subspecialist-specific factors should not be underestimated when designing guideline-based, computerized decision support systems for the subspecialty setting.
PMCID: PMC3279612  PMID: 22204897
Decision Support Systems; Clinical; Qualitative Evaluation; Asthma
3.  Bridging the Guideline Implementation Gap: A Systematic, Document-Centered Approach to Guideline Implementation 
Objective: A gap exists between the information contained in published clinical practice guidelines and the knowledge and information that are necessary to implement them. This work describes a process to systematize and make explicit the translation of document-based knowledge into workflow-integrated clinical decision support systems.
Design: This approach uses the Guideline Elements Model (GEM) to represent the guideline knowledge. Implementation requires a number of steps to translate the knowledge contained in guideline text into a computable format and to integrate the information into clinical workflow. The steps include: (1) selection of a guideline and specific recommendations for implementation, (2) markup of the guideline text, (3) atomization, (4) deabstraction and (5) disambiguation of recommendation concepts, (6) verification of rule set completeness, (7) addition of explanations, (8) building executable statements, (9) specification of origins of decision variables and insertions of recommended actions, (10) definition of action types and selection of associated beneficial services, (11) choice of interface components, and (12) creation of requirement specification.
Results: The authors illustrate these component processes using examples drawn from recent experience translating recommendations from the National Heart, Lung, and Blood Institute's guideline on management of chronic asthma into a workflow-integrated decision support system that operates within the Logician electronic health record system.
Conclusion: Using the guideline document as a knowledge source promotes authentic translation of domain knowledge and reduces the overall complexity of the implementation task. From this framework, we believe that a better understanding of activities involved in guideline implementation will emerge.
PMCID: PMC516249  PMID: 15187061
4.  Accuracy of a computerized clinical decision-support system for asthma assessment and management 
To evaluate the accuracy of a computerized clinical decision-support system (CDSS) designed to support assessment and management of pediatric asthma in a subspecialty clinic.
Cohort study of all asthma visits to pediatric pulmonology from January to December, 2009.
CDSS and physician assessments of asthma severity, control, and treatment step.
Both the clinician and the computerized CDSS generated assessments of asthma control in 767/1032 (74.3%) return patients, assessments of asthma severity in 100/167 (59.9%) new patients, and recommendations for treatment step in 66/167 (39.5%) new patients. Clinicians agreed with the CDSS in 543/767 (70.8%) of control assessments, 37/100 (37%) of severity assessments, and 19/66 (29%) of step recommendations. External review classified 72% of control disagreements (21% of all control assessments), 56% of severity disagreements (37% of all severity assessments), and 76% of step disagreements (54% of all step recommendations) as CDSS errors. The remaining disagreements resulted from pulmonologist error or ambiguous guidelines. Many CDSS flaws, such as attributing all ‘cough’ to asthma, were easily remediable. Pediatric pulmonologists failed to follow guidelines in 8% of return visits and 18% of new visits.
The authors relied on chart notes to determine clinical reasoning. Physicians may have changed their assessments after seeing CDSS recommendations.
A computerized CDSS performed relatively accurately compared to clinicians for assessment of asthma control but was inaccurate for treatment. Pediatric pulmonologists failed to follow guideline-based care in a small proportion of patients.
PMCID: PMC3078658  PMID: 21486882
Guidelines; controlled natural language; implementation; machine learning; communication; decision support; safety; informatics
5.  Information Technology for Children's Health and Health Care 
In September 2000, the Agency for Healthcare Quality and Research and the American Academy of Pediatrics Center for Child Health Research sponsored a meeting of experts and knowledgeable stakeholders to identify 1) the special information needs of pediatric care and 2) health service research questions related to the use of information technology in children's health care. Technologies that support the care of children must address issues related to growth and development, children's changing physiology, and the unique diseases of children and interventions of pediatric care. Connectivity and data integration are particular concerns for child health care workers. Consumer health information needs for this population extend beyond the needs of one individual to the needs of the family. Recommendations of the attendees include rapid implementation of features in electronic health information systems that support pediatric care and involvement of child health experts in policy making, standards setting, education, and advocacy. A proposed research agenda should address both effectiveness and costs of information technology, with special consideration for the needs of children, the development and evaluation of clinical decision support in pediatric settings, understanding of the epidemiology of iatrogenic injury in childhood, supplementation of vocabulary standards with pediatrics-specific terminology, and improvement in health care access for children, using telemedicine.
PMCID: PMC130065  PMID: 11687562
6.  GEM 
Objective: To develop a guideline document model that includes a sufficiently broad set of concepts to be useful throughout the guideline life cycle.
Design: Current guideline document models are limited in that they reflect the specific orientation of the stakeholder who created them; thus, developers and disseminators often provide few constructs for conceptualizing recommendations, while implementers de-emphasize concepts related to establishing guideline validity. The authors developed the Guideline Elements Model (GEM) using XML to better represent the heterogeneous knowledge contained in practice guidelines. Core constructs were derived from the Institute of Medicine's Guideline Appraisal Instrument, the National Guideline Clearinghouse, and the augmented decision table guideline representation. These were supplemented by additional concepts from a literature review.
Results: The GEM hierarchy includes more than 100 elements. Major concepts relate to a guideline's identity, developer, purpose, intended audience, method of development, target population, knowledge components, testing, and review plan. Knowledge components in guideline documents include recommendations (which in turn comprise conditionals and imperatives), definitions, and algorithms.
Conclusion: GEM is more comprehensive than existing models and is expressively adequate to represent the heterogeneous information contained in guidelines. Use of XML contributes to a flexible, comprehensible, shareable, and reusable knowledge representation that is both readable by human beings and processible by computers.
PMCID: PMC79044  PMID: 10984468
7.  Computer-based Guideline Implementation Systems 
In this systematic review, the authors analyze the functionality provided by recent computer-based guideline implementation systems and characterize the effectiveness of the systems. Twenty-five studies published between 1992 and January 1998 were identified. Articles were included if the authors indicated an intent to implement guideline recommendations for clinicians and if the effectiveness of the system was evaluated. Provision of eight information management services and effects on guideline adherence, documentation, user satisfaction, and patient outcome were noted.
All systems provided patient-specific recommendations. In 19, recommendations were available concurrently with care. Explanation services were described for nine systems. Nine systems allowed interactive documentation, and 17 produced paper-based output. Communication services were present most often in systems integrated with electronic medical records. Registration, calculation, and aggregation services were infrequently reported. There were 10 controlled trials (9 randomized) and 10 time-series correlational studies. Guideline adherence improved in 14 of 18 systems in which it was measured. Documentation improved in 4 of 4 studies.
PMCID: PMC61349  PMID: 10094063
8.  A Design Model for Computer-based Guideline Implementation Based on Information Management Services 
Clinical practice guidelines must be implemented effectively if they are to influence the behavior of clinicians. The authors describe a model for computer-based guideline implementation that identifies eight information management services needed to integrate guideline-based decision support with clinical workflow. Recommendation services determine appropriate activities in specific clinical circumstances. Documentation services involve data capture. Registration services integrate demographic and administrative data. Explanation services enhance the credibility of automated recommendations by providing supportive evidence and rating the quality of evidence. Calculation services measure time intervals, suggest medication dosages, and perform other computational tasks. Communication services employ standards for information transfer and provide data security. Effective presentation services facilitate understanding of complex data, clarify trends, and format written materials (including prescriptions) for patients. Aggregation services associate outcomes with specific guideline interventions. The authors provide examples of the eight services that make up the model from five evidence-based practice parameters developed by the American Academy of Pediatrics.
PMCID: PMC61348  PMID: 10094062
12.  Representation of Clinical Practice Guidelines in Conventional and Augmented Decision Tables 
Abstract Objective: To develop a knowledge representation model for clinical practice guidelines that is linguistically adequate, comprehensible, reusable, and maintainable.
Design: Decision tables provide the basic framework for the proposed knowledge representation model. Guideline logic is represented as rules in conventional decision tables. These tables are augmented by layers where collateral information is recorded in slots beneath the logic.
Results: Decision tables organize rules into cohesive rule sets wherein complex logic is clarified. Decision table rule sets may be verified to assure completeness and consistency. Optimization and display of rule sets as sequential decision trees may enhance the comprehensibility of the logic. The modularity of the rule formats may facilitate maintenance. The augmentation layers provide links to descriptive language, information sources, decision variable characteristics, costs and expected values of policies, and evidence sources and quality.
Conclusion: Augmented decision tables can serve as a unifying knowledge representation for developers and implementers of clinical practice guidelines.
PMCID: PMC61256  PMID: 9292844
16.  How “Should” We Write Guideline Recommendations? Interpretation of Deontic Terminology in Clinical Practice Guidelines: Survey of the Health Services Community 
Quality & safety in health care  2010;19(6):509-513.
To describe the level of obligation conveyed by deontic terms (words such as “ should,” “may,” “must,” and “is indicated”) commonly found in clinical practice guidelines.
Cross sectional electronic survey.
Researchers developed a clinical scenario and presented participants with recommendations containing 12 deontic terms and phrases.
All 1332 registrants of the 2008 annual conference of the US Agency for Healthcare Research and Quality.
Main Outcome Measures
Participants indicated the level of obligation they believed guideline authors intended by using a slider mechanism ranging from “No obligation” (leftmost position recorded as 0) to “Full obligation” (rightmost position recorded as 100.)
445/1332 registrants (36%) submitted the on-line survey. 254/445 (57%) reported they had experience developing clinical practice guidelines.133/445 (30%) indicated they provided healthcare. “Must” conveyed the highest level of obligation (median = 100) and least amount of variability (interquartile range = 5.) “May” (median = 37) and “may consider” (median = 33) conveyed the lowest levels of obligation. All other terms conveyed intermediate levels of obligation characterised by wide and overlapping interquartile ranges.
Members of the health services community believe guideline authors intend variable levels of obligation when using different deontic terms within practice recommendations. Ranking of a subset of terms by intended level of obligation is possible. Matching deontic terminology to intended recommendation strength can help standardise the use of deontic terminology by guideline developers.
PMCID: PMC2982946  PMID: 20702437
deontic; practice guidelines
17.  GEM at 10: A Decade’s Experience with the Guideline Elements Model 
The Guideline Elements Model (GEM) was developed in 2000 to organize the information contained in clinical practice guidelines using XML and to represent guideline content in a form that can be understood by human readers and processed by computers. In this work, we systematically reviewed the literature to better understand how GEM was being used, potential barriers to its use, and suggestions for improvement. Fifty external and twelve internally produced publications were identified and analyzed. GEM was used most commonly for modeling and ontology creation. Other investigators applied GEM for knowledge extraction and data mining, for clinical decision support for guideline generation. The GEM Cutter software—used to markup guidelines for translation into XML— has been downloaded 563 times since 2000. Although many investigators found GEM to be valuable, others critiqued its failure to clarify guideline semantics, difficulties in markup, and the fact that GEM files are not usually executable.
PMCID: PMC3243287  PMID: 22195106
18.  Clinical practice guideline development manual: A quality-driven approach for translating evidence into action 
Guidelines translate best evidence into best practice. A well-crafted guideline promotes quality by reducing healthcare variations, improving diagnostic accuracy, promoting effective therapy, and discouraging ineffective – or potentially harmful – interventions. Despite a plethora of published guidelines, methodology is often poorly defined and varies greatly within and among organizations.
This manual describes the principles and practices used successfully by the American Academy of Otolaryngology – Head and Neck Surgery to produce quality-driven, evidence-based guidelines using efficient and transparent methodology for action-ready recommendations with multi-disciplinary applicability. The development process, which allows moving from conception to completion in twelve months, emphasizes a logical sequence of key action statements supported by amplifying text, evidence profiles, and recommendation grades that link action to evidence.
As clinical practice guidelines become more prominent as a key metric of quality healthcare, organizations must develop efficient production strategies that balance rigor and pragmatism. Equally important, clinicians must become savvy in understanding what guidelines are – and are not – and how they are best utilized to improve care. The information in this manual should help clinicians and organizations achieve these goals.
PMCID: PMC2851142  PMID: 19464525
19.  Developing a Decision Support System for Tobacco Use Counseling Using Primary Care Physicians 
Informatics in primary care  2008;16(2):101-109.
Clinical decision support systems (CDSS) have the potential to improve adherence to guidelines, but only if they are designed to work in the complex environment of ambulatory clinics as otherwise physicians may not use them.
To gain input from primary care physicians in designing a CDSS for smoking cessation to ensure that the design is appropriate to a clinical environment before attempts to test this CDSS in a clinical trial. This approach is of general interest to those designing similar systems.
Design and Approach
We employed an iterative ethnographic process that used multiple evaluation methods to understand physician preferences and workflow integration. Using results from our prior survey of physicians and clinic managers, we developed a prototype CDSS, validated content and design with an expert panel, and then subjected it to usability testing by physicians, followed by iterative design changes based on their feedback. We then performed clinical testing with individual patients, and conducted field tests of the CDSS in two primary care clinics during which four physicians used it for routine patient visits.
The CDSS prototype was substantially modified through these cycles of usability and clinical testing, including removing a potentially fatal design flaw. During field tests in primary care clinics, physicians incorporated the final CDSS prototype into their workflow, and used it to assist in smoking cessation interventions up to eight times daily.
A multi-method evaluation process utilizing primary care physicians proved useful for developing a CDSS that was acceptable to physicians and patients, and feasible to use in their clinical environment.
PMCID: PMC2706833  PMID: 18713526
Smoking cessation; Medical Informatics; Qualitative Research; Guidelines
20.  Embedding the Guideline Elements Model in Web Ontology Language 
The Guideline Elements Model (GEM) uses XML to represent the heterogeneous knowledge contained in clinical practice guidelines. GEM has important applications in computer aided guideline authoring and clinical decision support systems. However, its XML representation format could limit its potential impact, as semantic web ontology languages, such as OWL, are becoming major knowledge representation frameworks in medical informatics. In this work, we present a faithful translation of GEM from XML into OWL. This translation is intended to keep the knowledge model of GEM intact, as this knowledge model has been carefully designed and has become a recognized standard. An OWL representation would make GEM more applicable in medical informatics systems that rely on semantic web. This work will also be the initial step in making GEM a guideline recommendation ontology.
PMCID: PMC2815420  PMID: 20351934
21.  Collaboration between the Medical Informatics Community and Guideline Authors: Fostering HIT Standard Development that Matters 
Clinical guideline authors, health information technology (HIT) standards development organizations, and information system implementers all work to improve the processes of healthcare, but have long functioned independently towards realizing these goals. This has led to clinical standards of care that often poorly align with the functional and technical HIT standards developed to realize them. We describe the shortcomings and inefficiencies inherent in this current process and introduce two national initiatives that attempt to unite these communities. The mission of these two initiatives is to create examples of unambiguous, decidable, and executable clinical guidelines which both utilize and inform HIT terminology and logical expression standards. All of the products of this work aim to facilitate enterprise-wide guideline implementation and create a rising tide which lifts all ships.
PMCID: PMC1839341  PMID: 17238298
22.  The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation 
Clinical practice guidelines are not uniformly successful in influencing clinicians' behaviour toward best practices. Implementability refers to a set of characteristics that predict ease of (and obstacles to) guideline implementation. Our objective is to develop and validate a tool for appraisal of implementability of clinical guidelines.
Indicators of implementability were identified from the literature and used to create items and dimensions of the GuideLine Implementability Appraisal (GLIA). GLIA consists of 31 items, arranged into 10 dimensions. Questions from 9 of the 10 dimensions are applied individually to each recommendation of the guideline. Decidability and Executability are critical dimensions. Other dimensions are Global, Presentation and Formatting, Measurable Outcomes, Apparent Validity, Flexibility, Effect on Process of Care, Novelty/Innovation, and Computability. We conducted a series of validation activities, including validation of the construct of implementability, expert review of content for clarity, relevance, and comprehensiveness, and assessment of construct validity of the instrument. Finally, GLIA was applied to a draft guideline under development by national professional societies.
Evidence of content validity and preliminary support for construct validity were obtained. The GLIA proved to be useful in identifying barriers to implementation in the draft guideline and the guideline was revised accordingly.
GLIA may be useful to guideline developers who can apply the results to remedy defects in their guidelines. Likewise, guideline implementers may use GLIA to select implementable recommendations and to devise implementation strategies that address identified barriers. By aiding the design and operationalization of highly implementable guidelines, our goal is that application of GLIA may help to improve health outcomes, but further evaluation will be required to support this potential benefit.
PMCID: PMC1190181  PMID: 16048653
23.  A draft framework for measuring progress towards the development of a national health information infrastructure 
American public policy makers recently established the goal of providing the majority of Americans with electronic health records by 2014. This will require a National Health Information Infrastructure (NHII) that is far more complete than the one that is currently in its formative stage of development. We describe a conceptual framework to help measure progress toward that goal.
The NHII comprises a set of clusters, such as Regional Health Information Organizations (RHIOs), which, in turn, are composed of smaller clusters and nodes such as private physician practices, individual hospitals, and large academic medical centers. We assess progress in terms of the availability and use of information and communications technology and the resulting effectiveness of these implementations. These three attributes can be studied in a phased approach because the system must be available before it can be used, and it must be used to have an effect. As the NHII expands, it can become a tool for evaluating itself.
The NHII has the potential to transform health care in America – improving health care quality, reducing health care costs, preventing medical errors, improving administrative efficiencies, reducing paperwork, and increasing access to affordable health care. While the President has set an ambitious goal of assuring that most Americans have electronic health records within the next 10 years, a significant question remains "How will we know if we are making progress toward that goal?" Using the definitions for "nodes" and "clusters" developed in this article along with the resulting measurement framework, we believe that we can begin a discussion that will enable us to define and then begin making the kinds of measurements necessary to answer this important question.
PMCID: PMC1177954  PMID: 15953388
24.  Take Note(s): Differential EHR Satisfaction with Two Implementations under One Roof 
Objective: The aim of this study was to rigorously evaluate perceived differences in satisfaction with an electronic health record (EHR) between residents of two medical specialties who share the same health record, practice location, administration, and information technology support.
Design: A cross-sectional survey was used comparing user satisfaction between pediatrics residents and internal medicine residents in an academic practice.
Measurements: The survey was designed to measure baseline demographic characteristics, attitudes toward computers, general satisfaction with an EHR, and perceived practicality of use, variation from familiar practice, organizational support, and impact on delivery of care.
Results: Medicine subjects were similar to pediatrics subjects in baseline demographic characteristics. Satisfaction with the EHR implementation was very high for both sets of subjects, but internal medicine residents were significantly less likely to be satisfied with the EHR implementation (relative risk [RR] = 0.84, 95% confidence interval [CI] = 0.73–0.98) and considerably less likely to believe that their colleagues were satisfied with it (RR = 0.56, 95% CI = 0.41–0.77). The only surveyed characteristic independently predicting satisfaction was medical specialty (p = 0.04). Medicine subjects were less likely to believe template-based documentation improved their efficiency (RR = 0.64, 95% CI = 0.46–0.88). They were significantly more likely to believe the system had been designed to improve billing (RR = 1.50, 95% CI = 1.05–2.04) and not to improve patient care (RR = 0.61, 95% CI = 0.44–0.85).
Conclusion: The authors found a difference in satisfaction between internal medicine and pediatrics users of an EHR. Although many potential factors that influence satisfaction were similar between subjects in the two specialties, differences in previous experience may have influenced the results. Medicine residents had more previous experience with a different EHR implementation, which they may have perceived as superior to the one involved in this study. Pediatric residents had more previous experience with structured data entry prior to EHR implementation and more preventive care patient encounters for which structured data entry may be well suited. Since successful implementations generally require satisfied users, understanding what factors affect satisfaction can improve chances of a system's success.
PMCID: PMC305457  PMID: 14527978
25.  The Incorporation of Clinical Practice Guidelines for Glaucoma into an Ophthalmology Electronic Medical Record 
Clinical practice guidelines represent the best current thinking on the management of acute and chronic medical conditions. Unfortunately, the implementation of such guidelines in clinical practice has been difficult and problematic. Electronic medical records represent an opportunity to incorporate guideline recommendations without disturbing physician workflow. We have designed a schema for incorporating guidelines for glaucoma management into an ophthalmology EMR.
PMCID: PMC1560790  PMID: 16779402

Results 1-25 (30)