|Home | About | Journals | Submit | Contact Us | Français|
The Guideline Elements Model (GEM) was developed in 2000 to organize the information contained in clinical practice guidelines using XML and to represent guideline content in a form that can be understood by human readers and processed by computers. In this work, we systematically reviewed the literature to better understand how GEM was being used, potential barriers to its use, and suggestions for improvement. Fifty external and twelve internally produced publications were identified and analyzed. GEM was used most commonly for modeling and ontology creation. Other investigators applied GEM for knowledge extraction and data mining, for clinical decision support for guideline generation. The GEM Cutter software—used to markup guidelines for translation into XML— has been downloaded 563 times since 2000. Although many investigators found GEM to be valuable, others critiqued its failure to clarify guideline semantics, difficulties in markup, and the fact that GEM files are not usually executable.
The Guideline Elements Model (GEM) was first developed in 2000 to store and organize the heterogeneous knowledge and information contained in clinical practice guidelines. GEM provides an intermediate knowledge representation that permits natural language guidelines to be translated into a format that can be processed by computers. GEM uses XML to describe a comprehensive set of pertinent concepts, relationships between concepts, and attributes. The resulting guideline representation can be used for multiple purposes including incorporation into decision support systems, electronic guideline distribution, and guideline querying. The GEM Cutter editor was designed to facilitate markup of guideline text and to facilitate its translation into XML. It was intended for an array of guidelines users including developers, disseminators, implementers, quality appraisers, and end users.
GEM’s XML representation has been updated to accommodate evolving needs. GEM was initially described by an XML Document Type Definition in 2002. In 2006, the model was augmented and published as an XML Schema. Both representations underwent standardization by ASTM International.
Ten years after its introduction (1) we sought to systematically review the literature to identify how GEM was being used outside the laboratory where it was created and to identify barriers to its use and suggestions for how it might be improved.
In June 2010, with the assistance of a medical librarian, we conducted a literature search using three databases: PubMed, SCOPUS, and ISI Web of Knowledge. Search terms used were: ‘GEM AND guideline elements model’ which resulted in 54 articles (see Figure 1). To further expand our search we used the original publication in which GEM was described (1), and searched for articles with related citations, or articles in which the original paper was referenced. Non-English journals were excluded. This expanded search resulted in 322 articles. After eliminating duplicates, 159 articles remained. Because of our interest in diffusion of the model, the 12 articles that were authored by Yale Center for Medical Informatics (YCMI) collaborators were set aside to be analyzed separately. Of the 147 remaining articles 59 could not be accessed (i.e., articles that were not electronically retrievable as full texts or available locally) and 38 articles were not about GEM, i.e. GEM was not mentioned in the text or referenced in the bibliography (e.g., geology articles). This resulted in a final list of 50 articles which were analyzed by all the authors (Figure 1). Articles were categorized according to subject matter and critiques of GEM were extracted for further analysis (Table 2). We reviewed a sample of 20 papers to help define categories of how GEM was being used. These subject categories were iteratively refined dependent upon major categories identified in the 20 papers, and continuously evalulated as the remaining articles were reviewed.
In addition, we queried the GEM website registry to characterize the incident downloads of the GEM Cutter software over the past 5 years. GEM Cutter has been freely available after a simple registration process.
The Guideline Elements Model was used or referenced in 50 manuscripts arising from 8 countries: Italy, Israel, Canada, Australia, France, Sweden, Austria and the United States. Review of the website registry of GEM Cutter downloads revealed 563 incident downloads since 2006 (see Table 1), suggesting that GEM has been applied by many users beyond those who have published their work.
We identified 4 categories of use of GEM: (1) Ontology/Modeling refers to papers primarily concerned with the development of an ontology or a model; (2) Knowledge extraction/retrieval/NLP/data mining refers to articles that are concerned with the extraction of information from guidelines using natural language processing or data mining; (3) Clinical Decision support/guideline application includes articles concerned with the application of guidelines in a clinical decision support system; and (4) Guideline generation refers to articles concerned with the authoring of clinical guidelines. The categories were found to be inclusive for all the articles reviewed, but were not mutually exclusive.
The most common application of GEM was for modeling or ontology creation (14/50 articles, 13 in which the model was applied and 1 in which the tool was used). GEM was applied in six articles for knowledge extraction/data mining, and in two articles for clinical decision support/guideline knowledge application. Two papers described application of GEM-based tools for guideline generation. The remaining articles did not use GEM, (either considered GEM but chose not to use it, or only made reference to GEM).
We found 12 papers in which use of GEM was considered and rejected, 10 of which were considering its use for modeling/ontology creation. In most cases, the authors indicated the reason why GEM was not selected (see below). GEM was referenced without discussion in 15 review articles about modeling/ontology creation, guideline generation, or clinical decision support.
Publications from the collaborators at the Yale Center for Medical Informatics included 4 articles in which both the GEM model and GEM related tools were applied for knowledge extraction/data mining, and/or clinical decision support/guideline knowledge application (Table 2b). The GEM model alone was applied in 5 articles and GEM-related tools were applied in 4 articles. These original research articles explored GEM’s used in modeling, knowledge extraction, clinical decision support and guideline generation. One review paper used GEM as an example (2).
GEM has been praised as “the leader” in the document-centered approach to guideline knowledge representation (3). In contrast to the model-centered approach to guideline formalization—a conceptual model in which the relationship between the original document and the model is only indirect—the document-centered approach takes the original guideline text as a starting point and constrains the translation process toward congruence with the source knowledge.
Several publications praised the detailed specification of the processes used for authoring guidelines: “GEM is considerably richer than other systems in specifying details of the guideline development process (e.g. authors, purpose, intended audience, versions, and the consensus process)” (4); and GEM’s explicit modeling of the evidence level of each part of the guideline (5). Users also appreciated GEM’s support of knowledge components “The Guideline Elements Model (GEM) is chosen… since it supports a broad range of knowledge elements, is easily extendable (it is based on XML), and its mark-up can be easily read and understood by domain professionals” (6). In addition, guideline developers recognized GEM as useful for maximizing the validity and utility of their products. “Yale Center for Medical Informatics used specialized software (GEM-COGS transforms, GLIA) to appraise adherence of the draft guideline to methodologic standards, to improve clarity of recommendations, and to predict potential obstacles to implementation. These steps helped to further maximize the validity and utility of the resulting guideline”(7).
GEM-Cutter was identified as useful for preventing errors when transitioning to newer versions of guidelines “(GEM Cutter) might limit errors that result from forgetting to represent part of the narrative guideline or to copy part of the clinical algorithm or sidebars to the next version. A tool like GEM-Cutter could be used to mark-up narrative guidelines using GEM elements. The tool could be used to view unmarked parts of the guideline, thus aiding in limiting omission errors”(8).
Several publications offered specific criticisms of the model. Four themes emerged from these critiques.
Several authors pointed to a need for GEM to promote semantic clarification of narrative guideline text. One criticism was that the representation did not fulfill terminology needs (9–10), specifically, that “GEM does little to resolve ambiguities … the model is simply an abstraction of the guideline document” (10). Other authors expressed a desire that GEM include diagnosis and procedure codes, “GEM doesn’t provide adequate facilities for the coding of diagnoses, procedures etc.”(12), and suggested that GEM should allow for terminology standards (13). An additional criticism about GEM output was that the organization of the original text was not retained in the XML output.
The GEM Cutter XML editor applies a copy and paste metaphor to markup and parse original guideline text into GEM elements and create an XML document. Some authors noted that GEM’s markup process was time-consuming (14) and that there was substantial variation in the results (10)(15). The quantity of effort required for this markup process is directly related to the comprehensiveness desired of the final XML document.
Several authors critiqued GEM for not incorporating a formal computational model aimed at guideline execution. Critics noted the absence of control structures and temporal elements, which would allow the logic of a guideline to unfold over time: “[GEM] does not seem to support extended care over significant time periods, due to the lack of persistent-memory mechanisms, interaction with an electronic patient record, and complex control structures” (16); “[GEM] misses the logic of a guideline that unfolds over time” (17). In addition GEM was critiqued for not having workflow awareness: “…for electronic guidelines to be useful, the knowledge and logic components must be structured so that the semantic relationships between the knowledge components and the execution logic/workflow are clearly defined” (18).
Some authors claimed that GEM did not use attributes (“although GEM also tries to differentiate between “explicit” and “inferred” guideline text, it doesn’t consistently use XML concepts, especially XML attributes”) (12); was not hierarchical (19); and did not include the level of scientific evidence (“GEM doesn’t provide adequate facilities for the coding of diagnoses, procedures etc. or for adding essential information, such as the level of scientific evidence of a recommendation”) (12).
As GEM approaches the end of its first decade of use, we note that the Guideline Elements Model has been applied for a wide variety of purposes and achieved mention in a substantial number of research reports. We presume that there have been additional uses that have not resulted in publication, as suggested by the 921 downloads of GEM Cutter.
In reviewing the literature, we found a number of concerns that have been put forth regarding the model. Recognizing these critiques as potential limitations of the model we sought to categorize and understand them.
With respect to concerns about GEM’s inability to clarify guideline semantics, we note that a goal of the model was to improve transparency of the knowledge transformation process. GEM was intended to provide a standardized intermediate representation that categorizes and labels the original guideline content, but not interpret that text. Retaining the original text allows users to judge the accuracy of the transformed representation. Improving the clarity of the guideline content was seen as a separate process that is aided by a number of XSL transforms (collectively named EXTRACTOR transforms), which help to highlight vagueness and under-specification and provide a substrate to which standardized terminology and concept codes may be attached. In addition, each GEM element includes an attribute that indicates whether it is original text (“explicit”) or modified text (“inferred”).
As a declarative model, documents generated using GEM were not designed to be directly executable. Many guidelines do not define a sequence under which their recommendations are to be performed. Although control structures are not well developed in the model, the <algorithm> element contains <action>, <conditional>, <branch>, and <synchronization> subelements (20). In addition, the <link> element can be used to define a sequence of actions. Gershkovitch and colleagues demonstrated that a GEM-processed guideline can indeed be directly executed using an execution engine devised for that purpose, although they agree that the process is inefficient and ‘artificial’ (21).
The markup process is indeed time-consuming, particularly for those unfamiliar with the model. However, it has been shown that the time to perform markup diminished substantially with experience(22). To reduce the variation in markup, GEM incorporates a set of clear definitions for its elements (displayed in GEM Cutter) and the authors have worked with users to define a set of markup conventions that have increased consistency.
Finally, with respect to the comments characterized as “miscellaneous,” we note that GEM is a hierarchical model that incorporates several attributes (including source, unique identifier,and language) in every element definition. In addition, the model includes multiple elements that define the level of scientific evidence including evidence.quality and recommendation.strength at the level of individual recommendations and Description.evidence.collection, Method.evidence.collection, Number.source.documents, Evidence.time.period, Method.evidence.grading; Rating.scheme; Method.evidence.combination; Specification.harm.benefit; Quantification.harm.benefit; Role.value.judgment ; and Role.patient.preference (1).
This study was limited in that we were unable to access electronically a substantial number of papers that were identified by our queries. It is likely that several of these papers were not relevant to this review, but it is possible that additional themes might have been uncovered and we may have had a degree of sampling bias
In summary, our review of published studies and GEM cutter downloads suggests that in its first decade of use, the Guideline Elements Model is being applied internationally. Understanding limitations of the model may help to set expectations about its use at a reasonable level. Overall GEM continues to lead the field of document-centered approaches to guideline modeling and knowledge transformation.
We are grateful for the support of the Agency for Healthcare Quality and Research for its support through contract HHSA 290200810011 and the National Library of Medicine for its support through 2R01LM007199 and Training grant T-15LM07065.