PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of wtpaEurope PMCEurope PMC Funders GroupSubmit a Manuscript
 
Yearb Med Inform. Author manuscript; available in PMC 2010 April 23.
Published in final edited form as:
PMCID: PMC2858818
EMSID: UKMS29401

Free and Open Source Enabling Technologies for Patient-Centric, Guideline-Based Clinical Decision Support: A Survey

Summary

Objectives:

Guideline-based clinical decision support is an emerging paradigm to help reduce error, lower cost, and improve quality in evidence-based medicine. The free and open source (FOS) approach is a promising alternative for delivering cost-effective information technology (IT) solutions in health care. In this paper, we survey the current FOS enabling technologies for patient-centric, guideline-based care, and discuss the current trends and future directions of their role in clinical decision support.

Methods:

We searched PubMed, major biomedical informatics websites, and the web in general for papers and links related to FOS health care IT systems. We also relied on our background and knowledge for specific subtopics. We focused on the functionalities of guideline modeling tools, and briefly examined the supporting technologies for terminology, data exchange and electronic health record (EHR) standards.

Results:

To effectively support patient-centric, guideline-based care, the computerized guidelines and protocols need to be integrated with existing clinical information systems or EHRs. Technologies that enable such integration should be accessible, interoperable, and scalable. A plethora of FOS tools and techniques for supporting different knowledge management and quality assurance tasks involved are available. Many challenges, however, remain in their implementation.

Conclusions:

There are active and growing trends of deploying FOS enabling technologies for integrating clinical guidelines, protocols, and pathways into the main care processes. The continuing development and maturation of such technologies are likely to make increasingly significant contributions to patient-centric, guideline-based clinical decision support.

Keywords: Guideline-based clinical decision support, free and open source, knowledge management, representation, standards, electronic health records, clinical guideline, modeling

Introduction

Clinical practice guidelines (CPGs) are broadly defined as: “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” [1]. Together with localized protocols and executable pathways, which in this paper will be included in the broad definition, CPGs are advocated to be important measures for reducing cost and improving quality of care. There are growing interests and active research in computerized CPGs to facilitate effective implementation in the complex clinical environments [2, 3].

While computerized clinical guidelines are considered to be critical components of evidence-based medicine [4-6], their true potential in improving health outcomes is yet to be fully realized; none of the research systems is in wide-spread use [7]. The main challenges are in integrating the guideline models into the clinical workflow to generate patient-centric, personalized recommendations and actions. To facilitate such integration, advanced tools and techniques are needed to support modeling, editing, and executing CPGs in the health care information technology (IT) and clinical decision support systems. These tools need to be simple to use, easily available, and work with different information systems in changing environments. In other words, they need to be affordable, interoperable, and adaptable; such requirements are difficult to meet in the traditional closed, proprietary, vendor- and application-specific health care IT models.

With the fast emergence of free and open-source (FOS) software systems that are technically sophisticated and open source business models that are economically viable [8], there are increasing evidences that demonstrate their promises in health care IT and clinical decision support systems [9, 10]. The FOS approach to software development1, in particular, is characterized by its 1) licensing model, in which the source code is available, and different licenses are defined in terms of derivative works, applications, and other issues; 2) usage philosophy, where users have freedom to copy, use and distribute the software, with different costing models; 3) development model, in which a community of developers work together to produce and feedback on frequent releases of the software; and 4) business model, where the revenues are mainly from support services, consulting services, and product enhancements [9, 11]. The success factors are mainly attributed to the success derived from the community development approach – meritocracy, division of labor, involvement of users, strong leadership - which is considered to be similar to the evidence-based model in health care [9]. The inherent advantages include promotion of standards, flexibility, options, public and private partnerships and community funding. These, incidentally, well match the requirements for the enabling technologies for guideline-based decision support.

In this paper, we survey the availability and functionalities of existing and the emerging FOS guideline modeling tools, tying in with an examination of the FOS supporting technologies for implementing terminology, data interchange, and electronic health record (EHR) standards that form the bases for patient-centric, guideline-based clinical decision support. We also summarize the current trends in the development and application of such methods and tools, and suggest some possible future directions.

Free and Open Source Software in Health Care

While research on the underlying methodologies started in the 1970s, history of FOS systems in health care knowledge management and quality assurance spans less than a decade. Some early projects have failed mainly due to the lack of support from community developers, and the relative uncertainty of the open source business model. Many different projects, however, have since emerged, some of them are widely adopted and successfully implemented in the health care settings. Various recent reports from the global scientific and business communities have advocated the open source approach as a viable and promising solution to foster public-private partnerships and promote interoperability in health care IT and clinical decision support solutions [9, 12]. Many Open-Source task forces, working groups, and scientific sessions have begun to emerge and are actively growing in leading health and biomedical informatics organizations, including the International Medical Informatics Association (IMIA), the American Medical Informatics Association (AMIA), and their regular meetings and conferences. Various public-private collaboration initiatives such as the Open Research Collaboration Principle [13, 14] and the Open Health Information Project [15], etc., and many organization meetings such as the recent Linuxworld Healthcare Day [16] and others for developers, vendors, users of open source health care IT to explore new collaborations are on-going. Compilations of FOS health care IT and clinical decision support software are widely available, including the list of open source healthcare software on Wikipedia [17], OpenClinical [18], and LinuxMedNews [19].

Knowledge Management Tools

Clinical decision support functions can be categorized along the main areas of context, knowledge and data source, decision support, information delivery, and workflow [20]. To be successfully deployed in the complex health care IT environments, computerized CPGs and other major clinical decision support technologies need to be built on the bases of common knowledge management platforms that include standards for terminology, data interchange, and electronic health records (EHRs). These standards are active research and development areas in the main foci of the European and United States coordinated efforts for standardization2 [21].

Implementing Terminology Standards

To successfully integrate guideline models into clinical decision support systems, the models and the systems need to be able to share common medical terminologies to represent the relevant concepts. Common medical terminology systems relevant in this context include those with underlying reference information or semantic models, because the guideline models often deal with classes of diseases, signs and symptoms, procedures, and their interrelations. Some representative terminology systems include Systematic Nomenclature of Medicine – Clinical Terms (SNOMED-CT) [22], Logical Observations, Identifiers, Names, and Codes (LOINC) [23], the Unified Medical Language System (UMLS) [24] and General Architecture for Languages, Encyclopedias, and Nomenclatures in Medicine (GALEN) [25]. SNOMED-CT requires a license to use. LOINC is free but not open-source. UMLS, which is a meta-language subsuming many other terminology systems, is developed based on the FOS model and distributed freely. An open source foundation, OpenGALEN (www.openGALEN.org) freely distributes the GALEN reference model and works with vendors and developers to support its extension and application [21]. Implementation of these terminology systems in health care IT applications is not easy, due to the complexity, the evolving nature, and the deep context and domain knowledge that usually associate with each system. A list of the major FOS available to support terminology implementation is shown in Table 1.

Table 1
Free and open-source tools for supporting implementation of classification (terminology) systems for integration of computerized guidelines, protocols and pathways

The Protégé system is a general ontology editing framework which is also commonly used in guideline editing. Its functionalities will be described in more detail in a later section.

A variety of FOS supporting tools for UMLS are available, including the more general platforms such as language and vocabulary tools [26], knowledge source servers [27], spell checkers [28] and other Natural Language Processing tools. These tools manage lexical variation, tokenize and parse text strings, suggest spelling variants, and provide text-to-concept mapping capabilities [26]. There are also more application-specific systems such as the open-source, interactive Java-based system created in the MIMIC II project for rapid encoding of significant events in the ICU using UMLS/SNOMED-CT [29].

RELMA is the open source tools package that is available on the LOINC website. It includes Intelligent Mapper, which has been used extensively for lexical mapping into LOINC terminology [30-32].

Current FOS resources for the OpenGALEN framework includes a large open source description logic-based ontology for the medical domain, its Common Reference Model, and a comprehensive ontology development environment, called OpenKnoME. OpenKnoME includes support for collaborative authoring including full browse and debug, and for rapid iterative ontology prototyping and development through use of intermediate representations [33].

To facilitate interoperability, many FOS tools have been used and are being developed for ontology mapping among the various medical terminology systems [34]. C-OWL is a general ontology mapping system that extends the Web Ontology Language (OWL); it has been applied in medical ontology mapping [35]. MoST is a recent FOS tool for ontology mapping between OpenEHR archetypes in the dual model architecture for EHRs (to be described below) and SNOMED-CT concepts [36, 37]. A variety of FOS tools have also been developed to support visualization and navigation of terminology systems, e.g., TermViz [38], which could be used independently or together with the ontology mapping tools.

Implementing Data Interchange Standards

The most common data exchange standards used in health care IT are Health Level Seven (HL7) for general health information, Digital Imaging and Communications in Medicine (DICOM) for medical images, and ISO/IEEE 11073 (previously called Medical Information Bus) for medical devices. HL7 version 3 has evolved from a pure data interchange format to include a Reference Information Model and a suite of other standards for capturing the conceptual structure of health information systems. The HL7 Clinical Guidelines Special Interest Group has also adopted Arden syntax [39], a process oriented medical knowledge representation, as a standard to encode guideline-based decision making episodes. Both HL7 and DICOM are built on the FOS philosophy, therefore most of the enabling and editing tools are FOS software as well. There are also some free conformance test tools available for 11073. A list of the major FOS tools available to support data interchange implementation is shown in Table 2.

Table 2
Free and open-source tools for supporting implementation of data interchange standards for integration of computerized guidelines, protocols and pathways

The FOS tools that facilitate worldwide adoption of the HL7 standards include the following [40]: Mirth (http://www.mirthproject.org/ [cited April 2007]) is a Java-based, open-source cross platform HL7 interface engine. HAPI (http://hl7api.sourceforge.net [cited April 2007]) is an open source, object-oriented HL7 Application Programming Interface (API) that includes a set of Java tools for HL7 parsing and encoding to support connectivity and message handling. The Perl HL7 Toolkit (http://hl7toolkit.sourceforge.net/ [cited April 2007]) is an open-source HL7 Perl API for manipulating, sending and receiving HL7 messages, an implementation of a forking HL7 server and an HL7 queue daemon. There is a syntactic version of the toolkit, called Net_HL7, being maintained by the PEAR group (http://pear.php.net/package/Net_HL7 [cited April 2007]).

xHL7, the eXcessively Simple Health Level-7 Processing Platform (http://aurora.regenstrief.org/xhl7/ [cited April 2007]) is a set of tools that use Java/XML Technology, SAX, and TRAX to process HL7 version 2 messages.

FOS packages for storage, manipulation, validation, and communication of DICOM include the following: DICOM Toolkit (DCMTK) (http://dicom.offis.de/dcmtk.php.en [cited April 2007]) and GCD (http://www.creatis.insa-lyon.fr/Public/Gdcm/ [cited April 2007]) are collections of libraries and applications implementing parts of the DICOM standard. Dcm4che (http://www.dcm4che.org/ [cited April 2007]) is a collection of Java applications and utilities for clinical image and object management using DICOM and HL7 services. DICOM validation tools (DVT) (http://sourceforge.net/projects/dvt/ [cited April 2007]) is a set of utility and .NET components for testing and validating DICOM objects.

As compared to HL7 and DICOM, there are relatively few facilitating tools for 11073, partly due to its complexity. The National Institute of Standards and Technology (NIST) and the Integrating the Health Enterprise initiative (IHE, see below) are involved in developing a set of compliance testing tools, ICSgenerator, and validation testing tools, PDUvalidate, for 11073 in the IHE patient care device domain.

Implementing Electronic Health Record Standards

Several recent FOS initiatives for generic interoperable EHR standards have emerged by designing and developing organization and communication infrastructure for the effective management of health data and documents [41]. While some of these initiatives have evolved from terminology or data interchange standards work, e.g., the HL7 Reference Information Model (RIM) and Clinical Document Architecture (CDA), others are targeted at developing platform technologies that support a full range of storage, interface, security and decision support services in complex, distributed or federated EHR implementations. Some examples include the ongoing European Committee for Standardization (CEN) Technical Committee 251 (TC251) Health Informatics -EHR Communications Standard EN 13606 (EHRcom), the Eclipse Foundation's Open Healthcare Framework (OHF), OpenEHR, IHE's various models and services for EHRs in different domains, and the smaller scale Evidence Based Guideline and Decision Support System (EGADSS). These frameworks provide a range of FOS tools and systems for supporting EMR development, implementation, and incorporation of patient-centric, guideline-based decision support capabilities. A list of the major FOS available to support EHR standards implementation is shown in Table 3.

Table 3
Free and open-source platform technologies for implementation of electronic health records and integration with computerized guidelines, protocols and pathways

Efforts in this area generally adopt the dual model approach, which establishes a Reference Model and an Archetype Model. The Reference Model defines the basic information model of the EHRs, which describes common data structures with minimal semantics. The Archetype Model specifies the knowledge on formal representations of the clinical concepts in the Reference Model and how different components can be combined and instantiated for different tasks, applications, and interfaces [41, 42]. Such separation enables modular integration of clinical guidelines.

While not a full-fletch EHR architecture standard, the RIM and CDA standards in HL7 are moving toward comprehensive specifications for different components in a general EHR architecture and communication infrastructure. Different enabling tools for implementing the components are available in various HL7 tools repositories.

The OHF project from the Eclipse Foundation (http://www.eclipse.org/ohf [cited April 2007]) facilitates inter-operability by developing a set of extensible frameworks, facilitating tools, and information and exchange standards that can be modularly incorporated into different applications and systems. The OHF components include healthcare related standard protocols, data structures, encryption and security tools etc., that can be used to develop different EHRs, usually in client/server architectures. The OHF components could also be implemented as web services for the existing health care IT systems [43].

The CEN/TC251 EN13606 EHR standard (EHRcom) (http://www.centc251.org/ [cited May 2007]) adopts the dual model approach to establish a full set of protocols and services for Health informatics – EHR communication. There are four parts in this developing standard: Reference architecture, clinical domain modeling, security requirements and distribution rules, and exchange protocols.

The openEHR framework (http://www.openehr.org [cited April 2007]) includes specifications of information models and service models, and implementations for an open, integrated healthcare IT platform for interoperable EHRs [44]. The platform supports standard medical terminology and data interchange standards, and publishes the interface specifications in directly usable implementation technologies such as OMG, IDL, XML, programming languages, and databse schemas. It also facilitates integration with legacy health care IT systems through syntactic interoperability and semantic interpretability of the archetypes. [45].

The LinkEHR project (http://pangea.upv.es/linkehr/ [cited June 2007]) in Spain aims to develop a set of tools for accessing existing heterogeneous EHRs for patient information. The tools support transformation and standardization of clinical data using the dual model architecture by mapping archetype ontologies and definitions to the elements of data sources such as XML documents or relational data sources. In particular, the LinkEHR-ED [42] is a tool for developing integration archetypes by providing different interfaces for different types of users. Successful applications have been implemented for EHRcom and openEHR.

The IHE initiative (http://www.ihe.net [cited April 2007]) is a framework for integrating information systems in a healthcare environment. Originated from a joint effort between the Radiological Society of North America (RSNA) and the Healthcare Information and Management Systems Society (HIMSS) 1998, IHE has gained support globally. A number of technical and medical domain frameworks are available, including IT infrastructure, laboratory, cardiology, eye care, radiology, etc. The frameworks promote coordinated use of existing medical terminology and data exchange standards and establish detailed implementation guidelines as Integration Profiles on how to develop the components for an interoperable, integration health care IT system. A set of public domain test tools, e.g., MESA tools, and the ICSgenerator for ISO/IEEE 11073 compliance, are available for validating the integration results in different implementations.

Closer to guideline-based care, EGADSS (http://www.egadss.org [cited April 2007]) is an open source tool under the GNU Free Documentation License for centralized guideline management in primary care. It adopts a service-oriented architecture that would respond to requests from existing EHRs to provide patient specific point of care reminders based on its internal collection of guidelines. EGADSS interfaces with existing EHRs via a HL7 CDA Document exchange through platform-independent web service. A knowledge base consisting of rules written as Arden Syntax Medical Logic Modules (MLMs) performs logical reasoning on the patient information. EGADSS can be used as a basis as part of patient-centric, Alerting, Reminding, Assisting, and Diagnosing guideline systems [46].

Guideline Modeling Tools

A number of frameworks are available for directly implementing CPGs in a computer-interpretable and -executable format. Asbru, EON, GLIF, Guide, Prodigy, and PROforma, for example, have previously been reported [47] and compared [4]. These frameworks are tailored for specific classes of guidelines, specific users, and specific organizations. Each framework supports specific guideline representation languages. Most of these languages are sufficiently complex that the manual formalization of CPGs is a challenging, burdensome, and time-consuming task. Thus, various tools and techniques have been developed to support the CPG modeling process. These tools foster different approaches that can be roughly classified into model-centric and document-centric tools. Most of the computerized CPG modeling tools, however, are proprietary software that is free (of charge), while only a few have adopted the FOS approach for development and deployment.

Model-centric Tools

In model-centric approaches a conceptual CPG model is formulated by domain experts. The relationship between the model and the original paper document is only indirect.

Protégé

Protégé (http://protege.stanford.edu [cited May 2007]) is an open source ontology development and knowledge acquisition environment developed by Stanford Medical Informatics [48]. It is a Java tool, which provides an extensible architecture for the creation of customized knowledge-based tools and assists users in the construction of large electronic knowledge bases.

Protégé provides two main ways of modeling ontologies: (1) Protégé-Frames editor and (2) Protégé-OWL editor enabling users to build ontologies in the Web Ontology Language (OWL). In Protégé-Frames the knowledge model is compatible with the Open Knowledge Base Connectivity protocol (OKBC) [49]. Thereby, all entities (i.e., instances, classes, slots, facets, and constraints) are frames. Instances represent objects in the domain of interest. Classes are either named collections of instances or abstract conceptual entities in the domain (e.g., the concept of a drug ingredient). Slots are binary relations describing properties of classes (e.g., the indications of a drug). Facets describe properties of slots (e.g., the data type of a slot's value). Constraints specify additional relationships that must hold among instances.

Protégé supports the construction of a domain ontology, the design of customized knowledge-acquisition forms, and entering domain knowledge. It provides a platform which can be extended with graphical widgets for tables, diagrams, and animation components to access other knowledge-based systems embedded applications. Protégé is a library, which other applications can use to access and display knowledge bases. It can be extended by way of a plug-in architecture and a Java-based Application Programming Interface (API) for building knowledge-based tools and applications. Protégé is used to author guidelines in various models. Part of the modeling can be accomplished using predefined graphical symbols. These symbols are arranged in a diagram and linked by graphs (See Figure 1 for an example). The underlying data is entered by forms.

Fig. 1
View of Protégé being used to author a guideline for managing chronic cough. The guideline model being used in this application is Dharma, part of the EON framework.

Projects using Protégé

To represent data and knowledge in EON [50, 51] needed for guideline-based decision support several models are necessary: (1) an information model of patient data as a set of classes and attributes, (2) a medical-specialty model consisting of an information model of the structure of domain concepts and relationship, an ontology of domain concepts organized into taxonomic hierarchies, and (3) a guideline ontology/information model called Dharma that defines the structure of the guideline knowledge required for performing guideline-based application tasks. The Dharma guideline model (see Figure 1) includes criteria and query languages that use the patient-data model and medical-specialty model to specify relevant patient situations.

Protégé can be used for encoding and validating guidelines in GLIF3 [52]. The tool can be used by domain experts for creating abstract flowcharts. This allows a guideline author to specify clinical algorithms, codes for clinical terms, rules for ranking alternative treatment options written in natural language, and documentation attributes. Knowledge engineers can create detailed computable specifications important for assessing guideline validity.

HELEN [53] is a framework for the whole life cycle of CPGs. It provides both methods and tools for authoring and adaptation of the guidelines. For both steps Protégé is used to easily create a user interface. A Protégé ontology is used to represent CPGs and to export it to HELEN's XML-based format.

In the SAGE [54] project, Protégé is the development environment for creating the ontologies required for formalizing CPGs and the workbench for encoding these computer-interpretable guidelines as Protégé knowledge bases. Protégé can also be used to author and view guidelines in the Prodigy [55] or PROforma [56] models.

AsbruView

AsbruView (http://www.asgaard.tuwien.ac.at/tools/asbruview.html [cited May 2007]) is a graphical user interface developed in the Asgaard/Asbru project to support the development of guidelines and protocols in Asbru [58].

Asbru is a complex language, which cannot be fully understood by physicians who have no or hardly any training in formal methods. AsbruView is a tool to make Asbru accessible to physicians, and to give any user an overview of a plan hierarchy. AsbruView is based on visual metaphors to make the underlying concepts easier to grasp. Currently, AsbruView provides four views: Topological View (see Figure 2), Temporal View (see Figure 3), SOPOView (see Figure 4), and XML View.

Fig. 2
AsbruView: Topological View
Fig. 3
AsbruView: Temporal View
Fig. 4
AsbruView: SopoView

The metaphors and graphical representation of AsbruView have proved to be useful in communicating Asbru's concepts to physicians. Users get a better overview of the therapy steps than from tables, while at the same time being able to see the precise temporal constraints of plans (which is not the case with flowcharts).

TALLIS

The TALLIS Toolset (http://www.acl.icnet.uk/lab/tallis/index.html [cited May 2007]) [59] includes software and training materials to create, publish, and enact clinical knowledge applications over the web. It is developed by the Advanced Computation Laboratory at Cancer Research UK and built on the PROforma language [60, 61] for modeling clinical processes. The toolset consists of three components that interact with each other: the Composer, the Tester, and the Engine.

PROforma tasks (i.e., plans, decisions, actions, enquiries) can be connected up to form a network. Such a network is sometimes called a “workflow”. The Composer provides a graphical interface to support the generation of such task networks. The development of a network is a two-step process: (1) a high level structure of the process is laid out and assembled as a network; (2) detailed knowledge that is required to enact each component task is entered as task attributes (see Figure 5). The Tallis Tester is a tool for testing and debugging the logic of a developed PROforma application. A tested and debugged application can then be enacted by the Tallis Engine.

Fig. 5
Tallis Composer [62]. The left pane presents the tree view that shows the process-description's hierarchical structure. The network view displays task ordering according to scheduled constraints. Task attributes can be entered in the bottom panes.

Document-centric Tools

Markup-based tools for editing and modeling computerized CPGs utilize a document-centric approach. The original guideline document is systematically marked-up by the user in order to generate a semi-formal model of the marked text part.

GEM Cutter

The GEM Cutter (http://gem.med.yale.edu/GEM_Cutter/gem_cutter.htm [cited April 2007]) [63] developed by Yale Center for Medical Informatics facilitates the transformation of CGPs into the Guideline Elements Model (GEM) [64], which is an XML-based guideline document model.

GEM Cutter's main screen consists of three vertical segments (see Figure 6) showing the original text of the guideline, a tree view of the developing GEM file, and any additional, important information about the GEM file. The translation of the knowledge contained in guideline text into a computable format and the integration of the information into clinical workflow requires a number of steps [66] (e.g., markup of the guideline text, atomization, de-abstraction and disambiguation of recommendation concepts, verification of rule set completeness, addition of explanations), whereas some of them are accomplished by GEM Cutter.

Fig 6
GEM Cutter [65]. The left pane shows the Guideline Text Segment, the middle pane shows the GEM Tree Segment, and the right pane shows the Element segment.

Document Exploration and Linking Tool / Addons (DELT/A)

The Institute of Software Technology and Interactive Systems at the Vienna University of Technology has developed a tool to provide a relatively easy way to translate free text into various (semi-) formal, XML-based representations. The open source tool DELT/A (http://ieg.ifs.tuwien.ac.at/projects/delta/ [cited May 2007]) [67] (formerly known as Guideline Markup Tool – GMT) provides two main features: (1) linking between a textual guideline and its formal representation, and (2) applying design patterns in the form of macros.

DELT/A's user interface (see Figure 7) consists of three panes. The top left and right panes provide equivalent views to either edit XML files or HTML files. The Macros pane provides either a structure view, search view, or insertable macros view, as well as a preview of the current macro.

Fig 7
Document Exploration and Linking Tool / Addons

DELT/A allows the definition of links between the original guideline and the target representation, which gives the user the possibility to find out where a certain value in the XML-language notation comes from. Using macros allows creating and extending specific XML files more easily through the usage of common design patterns. A macro combines several XML elements, which are usually used together. Such templates can be easily defined for any XML language. Together with the underlying schema DELT/A can interpret and insert them with the information necessary at the correct position within the document.

Therefore, DELT/A supports not only the authoring and augmenting of guidelines, but also the understanding of the (semi-)formal representation of the guideline as well as the structuring of the syntax of the (semi-)formal representation using its macro features. Furthermore, the original text parts need not be stored as part of the target representation elements. The links clearly show the source of each element in the target representation. There is no need to produce a guideline in natural language from the target representation since the original text remains unaltered.

Projects using DELT/A

DELT/A is used in several projects to formalize documents. Originally, it was developed to support the modeling of CPGs in Asbru. But soon it was obvious that a direct transformation from the textual guideline to an Asbru guideline was too difficult to accomplish; several approaches using intermediate representations were developed to finally achieve the Asbru format of a CPG.

The many-headed bridge (MHB) between guideline formats [68] bridges the gap between informal representations such as free text and tables and more formal representations such as Asbru, GLIF, or PROforma. The overall structure of an MHB file is a series of chunks corresponding to a certain bit of information in the natural language guideline text (i.e., a sentence, part of a sentence, more than one sentence). The information in a chunk is structured in various dimensions (i.e., control flow, data flow, temporal aspects, evidence, background information, resources, patient related aspects, structure). The translation to MHB is performed with DELT/A (see also Figure 7). Special macros developed help defining the MHB models.

Another transformation method using DELT/A is LASSIE (modeLing treAtment proceSSes using Information Extraction) [69, 70]. It automates parts of the modeling task generating intermediate representations stepwise by using Information Extraction techniques. Each intermediate representation can be visualized and reviewed using DELT/A as each model transformed is connected with the corresponding models in the previous representation formats.

Taboada et al. [71] applied the CommonKADS methodology to obtain a compact knowledge model of clinical guidelines. As guidelines contain incomplete and ambiguous knowledge, they further proposed a method to detect what parts of guideline knowledge are missing. They established explicit mappings between the guideline document and the knowledge model that can be viewed with the DELT/A tool to reduce the gap between the original representation and the corresponding knowledge model.

Stepper

Stepper (http://euromise.vse.cz/stepperen/index.php [cited May 2007]) [72] is a mark-up tool for narrative guidelines. The goals of the Stepper project are to develop both a stepwise method for formalization (in this context, XML transformation) of text documents of clinical guidelines and an XML editor enhanced with features to support this method. Stepper takes a guideline text as its starting point and splits the formalization process into multiple user-definable steps [73], each of which corresponds to an interactive XML transformation. The result of each step is an increasingly formalized version of the source document. An embedded XSLT processor carries out non-interactive transformation. Both the mark-up and the iterative transformation process are carried out by rules expressed in a new transformation language based on XML, the so-called XML Knowledge Block Transformation (XKBT).

By using the Stepper method and tool it is possible to transform CPGs into fragments of operational code (e.g., Java) or into parts of a guideline representation language (e.g., Asbru). Stepper's main advantage is the documentation of all activities, which allows easy review of the transformation process. Stepper also provides an interface showing the interconnection between the source text and the model.

Uruz

Uruz (http://medinfo.ise.bgu.ac.il/MedLab/ResearchProjects/RP_Uruz.htm [cited June 2007]) is a web-based guideline markup tool within the Degel [74] framework. Degel supports guideline classification, semantic markup, context-sensitive search, browsing, runtime application, and retrospective quality assessment. While currently supporting Asbru, GEM, and GLIF, it is applicable for any XML-based guideline representations.

Uruz supports the gradual migration of free text guidelines to formal representations, but it can also be used to create a guideline document de-novo (i.e., without using any source) by directly writing into the knowledge roles of a selected target ontology. The editor (see Figure 8) can modify the current content or add new content. This enables implicit knowledge to become more explicit, further facilitating the knowledge engineering task of fully formalizing the guideline.

Fig 8
Uruz web-based markup tool [75]

Several features are especially tailored to Asbru, such as the plan-body wizard (PBW), which is used for defining the guideline's control structure. The PBW enables the user to decompose actions embodied in the guideline into atomic actions and other sub-guidelines, and to define the control structure relating to them (e.g., sequential, parallel, repeated application).

Furthermore, Uruz enables the user to embed in the guideline document terms originating from standard vocabularies (i.e., ICD-9-CM, CPT-4, LOINC-3) from the integrated vocabulary server.

Conclusion

A plethora of FOS enabling technologies based on standards and consensus are increasingly available and showing great promises in supporting integration of CPGs into the clinical workflow. The main challenge, however, remains in the computerized modeling of the CPGs that would generate comprehensive, accurate, and targeted action recommendations that can be customized at the point-of-care. While a variety of sophisticated, formal CPG representations and languages are available, most of the modeling and editing tools are proprietary, albeit free, and have limited acceptance and usage in practice due to the complexity of the modeling tasks. Different approaches, however, are being explored to ease these tasks. One of the main observations from the FOS movement in IT in general, and in health care IT in particular, is that the community model, with active participation and feedback from various stakeholders – developers, users, and vendors, can be effective in promoting standards and disseminating advanced technologies and applications. As a successful, FOS guideline modeling framework, the evolution of Protégé has largely been the result of the enthusiasm and dedication of its active user community, which supports the development team while contributing new software components of its own [76]. On the other hand, target users, especially clinical domain experts, have not been actively involved in the development of most existing, non-FOS guideline modeling tools. As a result, these tools usually cannot be readily used by the actual CPG authors.

While the guideline modeling tools surveyed in this paper are important instruments toward integrating CPGs into the main care processes, widely acceptable common standards for computerized guideline representation formalisms or guideline modeling tools are yet to be established [77]. There are, however, growing trends to adopt terminology, communication, and architecture standards in developing guideline implementation workbenches. The SAGE project, for example, mirrors the emerging FOS EHR platforms discussed earlier by involving vendors, adopting standards such as UMLS, and defining a set of information models to facilitate interfacing between guideline representation and implementation [7]. In terms of functionalities, development of the guideline modeling techniques and tools are moving toward integrating both the model centric and document centric approaches. The main motivation is to identify and extract the relevant content of the paper-based guidelines to establish initial formal guideline models using the document-based tools; the models can then be further improved or edited using the model-centric tools. This approach ensures that the formal guideline models are very close to the original guideline documents, often by providing links between the formal models and their corresponding parts in the paper-based guidelines. The resulting formal guideline model would also support generation of paper-based guidelines, and further updating and customization based on the patient-specific information in the integrated health care IT systems [78, 79]. Recent work in this area makes use of natural language processing [80] and other machine learning techniques [81] for information extraction from the paper-based guidelines.

There are active and growing trends in the development and deployment of FOS enabling technologies for effective implementation of computerized CPGs. Past failures have shed new insights about the requirements for successful implementation of such systems. Recent successes have illuminated promising trends on the models for viable collaboration among different stake holders. The continuing development and maturation of such technologies are likely to make increasingly significant contributions to patient-centric, guideline-based clinical decision support.

Acknowledgment

Part of this work is supported by “Fonds zur Förderung der wissenschaftlichen Forschung FWF” (Austrian Science Fund), grant L290-N04.

Footnotes

1The notion of “Free” here refers to “Freedom to use”; it is different from the other notion of “Free software” that costs zero in price but without making the source code available. The latter is usually proprietary software which is free of charge. A comprehensive differentiation of the different software licensing models [11] is available at: http://www.fsf.org/licensing/essays/free-sw.html [cited April 2007].

2We do not separately address standards for security, privacy, and confidentiality in this survey.

References

1. Field MJ, Lohr KN, editors. Guidelines for Clinical Practice: From Development to Use. National Academy Press; Washington, DC: 1992.
2. Timmermans S, Mauck A. The Promises And Pitfalls Of Evidence-Based Medicine. Health Aff. 2005;24(1):18–28. [PubMed]
3. Clancy CM, Cronin K. Evidence-Based Decision Making: Global Evidence, Local Decisions. Health Aff. 2005;24(1):151–62. [PubMed]
4. Peleg M, Tu SW, Bury J, Ciccarese P, Fox J, Greenes RA, et al. Comparing Computer-Interpretable Guideline Models: A Case-Study Approach. J Am Med Inform Assoc. 2003;10(1):52–68. [PMC free article] [PubMed]
5. Wang D, Peleg M, Tu S, Boxwala A, Greenes R, Patel V. Representation primitives, process models and patient data in compueter-interpretable clinical practice guidelines: A literature review of guideline representation methods. Int J Med Inform. 2002;68(1/3):59–70. [PubMed]
6. de Clercq PA, Blomb JA, Korsten HHM, Hasman A. Approaches for creating computer-interpretable guidelines that facilitate decision support. Artif Intell Med. 2004;31:1–27. [PubMed]
7. Sonnenberg FA, Hagerty CG. Computer-interpretable clinical practice guidelines: Where are we and where are we going? In: Kulikowski C, Haux R, editors. IMIA Yearbook of Medical Informatics 2006. Methods Inf Med. Vol. 45. 2006. pp. 145–58. Suppl 1. [PubMed]
8. Riehle D. The Economic Motivation of Open Source Software: Stakeholder Perspectives. IEEE Computer. 2007 Apr;:25–32.
9. Goulde M, Brown E. Open Source Software: A Primer for Health Care Leaders. Report by Forrester Research. 2006. [cited March 2007]; http://www.chcf.org/topics/view.cfm?itemID=119091.
10. Preimesberger C. New Software Predicts Spread of Infectious Diseases. eWeek 2007 June 12, 2007 [cited 2007 June 14]; http://www.eweek.com/article2/0,1895,2145254,00.asp.
11. The Free Software Definition Feb 12, 2005. [cited 2007 May]; http://www.fsf.org/licensing/essays/free-sw.html.
12. Kalra D, Forslund D. Open source health systems. In: Demetricades JE, Kolodner RM, Christoperson GA, editors. Person-Centred Health Records: toward HealthePeople. Springer; New York, USA: 2005. pp. 169–85.
14. Understand the emerging innovation landscape Enabling university and industry research. 2006. [cited 2007 April]; http://www-304.ibm.com/jct09002c/university/scholars/downloads/Open_Collaboration_Principles_overview.pdf.
15. OHIP - Open Health Information Project Mar, 2007. [cited 2007 April]; http://osuosl.org/node/59.
16. Linuxworld Healthcare Day 2006. [cited 2007 June]; http://www.linuxworldexpo.com/live/12/events/12SFO06A/conference/CC730223.
17. Wikipedia - The Free Encyclopedia List of Open Source Healthcare Software. Jun 11, 2007. [cited 2007 June]; Available from: http://en.wikipedia.org/wiki/List_of_open_source_healthcare.
18. OpenClinical Open Source methods, tools and applications. Nov 21, 2006. [cited 2007 June]; http://www.openclinical.org/opensourceDLD.html.
19. LinuxMedNews 2007. [cited 2007 June]; http://www.linuxmednews.org/
20. Berlin A, Sorani M, Sim I. A taxonomic description of computer-based clinical decision support systems. J Biomed Inform. 2006;39(6):656–67. [PubMed]
21. Hammond WE, Cimino JJ. Standards in Biomedical Informatics. In: Shortliffe EH, Cimino JJ, editors. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. 3rd ed Springer; 2006. pp. 698–736.
22. Coté R, Rothwell D, Palotay J, Beckett R, Brochu L, editors. The Systematized Nomenclature of Medicine. College of American Pathologists: SNOMED International; Northfield, IL: 1993.
23. Huff SM, Rocha RA, McDonald CJ, De Moor GJ, Fiers T, Bidgood WDJ, et al. Development of the Logical Observation Identifier Names and Codes (LOINC) vocabulary. J Am Med Inform Assoc. 1998;5(3):276–92. [PMC free article] [PubMed]
24. Humphreys BL, Lindberg DAB. The UMLS project: making the conceptual connection between users and the information they need. Bull Med Libr Assoc. 1993;81(2):170–8. [PMC free article] [PubMed]
25. Rector AL, Nowlan WA. The GALEN Project. Computer Methods and Programs in Biomedicine. 1994;45(1/2):75–8. Special Issue on the AIM Projects. [PubMed]
26. Browne AC, Divita G, Aronson AR, McCray AT. UMLS Language and Vocabulary Tools. In: Musen MA, editor. Proceedings of the 2003 AMIA Annual Symposium; Washington, DC. 2003. p. 798. [PMC free article] [PubMed]
27. Bangalore A, Thorn KE, Tilley C, Peters L. The UMLS Knowledge Source Server: An Object Model For Delivering UMLS Data. In: Musen MA, editor. Proceedings of the 2003 AMIA Annual Symposium; Washington, DC. 2003. pp. 51–55. [PMC free article] [PubMed]
28. Tolentino HD, Matters MD, Walop W, Law B, Tong W, Liu F, et al. A UMLS-based spell checker for natural language processing in vaccine safety. BMC Medical Informatics and Decision Making. 2007;7(3) 2007. [PMC free article] [PubMed]
29. Shu J, Clifford G, Saeed M, Long W, Moody G, Szolovits P, et al. Computers in Cardiology. IEEE Computer Society Press; 2004. An Open-Source, Interactive Java-Based System for Rapid Encoding of Significant Events in the ICU Using the Unified Medical Language System; pp. 197–200.
30. Khan A, Griffith S, Moore C, Russell D, Rosario A, Bertolli J. Standardizing Laboratory Data by Mapping to LOINC. J Am Med Inform Assoc. 2006;13(3):353–5. [PMC free article] [PubMed]
31. Sun JY, Sun Y. A System for Automated Lexical Mapping. J Am Med Inform Assoc. 2006;13(3):334–43. [PMC free article] [PubMed]
32. Vreeman DJ, McDonald CJ. Automated mapping of local radiology terms to LOINC. In: Friedman CP, Ash J, Tarczy-Hornoch P, editors. Proceedings of the 2005 AMIA Annual Symposium; Washington, DC. 2005. pp. 769–73. [PMC free article] [PubMed]
33. Rector AL, Rogers JE, Zanstra PE, Haring Evd. OpenGALEN: Open Source Medical Terminology and Tools. In: Musen MA, editor. Proceedings of the 2003 AMIA Annual Symposium; Washington, DC. 2003. [PMC free article] [PubMed]
34. van Harmelen F. Ontology mapping: a way out of the medical tower of Babel? [Keynote address at the 10th Conference on Artificial Intelligence in Medicine (AIME 05)]; Aberdeen, Scotland: IOS Press; 2005. Amsterdam, NL.
35. Stuckenschmidt H, Harmelen Fv, Serafini L, Bouquet P, Giunchiglia F. Using C-OWL for the Alignment and Merging of Medical Ontologies. In: Hahn U, editor. First International Workshop on Formal Biomedical Knowledge Representation (KRMed'04); Whistler, Colorado. 2004 June.2004. pp. 8–101.
36. Qamar R, Rector A. MoST: A System to Semantically Map Clinical Model Data to SNOMED-CT. Semantic Mining Conference on SNOMED. 2006:38–43.
37. Qamar R, Rector A. Semantic Issues in Integrating Data from Different Models to Achieve Data Interoperability. In: Kuhn KA, Warren JR, Leong TY, editors. MEDINFO 2007: Proceedings of the 12th World Congress on Medical Informatics; Brisbane, Australia: IOS Press; 2007. Amsterdam, NL. (To appear)
38. Sundvall E, Nyström M, Petersson H, Åhlfeldt H. Interactive visualization and navigation of complex terminology systems, exemplified by SNOMED-CT. In: Hasman A, Haux R, van der Lei J, De Clercq, Roger France F, editors. The 20th International Congress of the European Federation for Medical Informatics, Studies in Health Technology and Informatics; 2006; Maastricht, The Netherlands: IOS Press; 2006. pp. 851–6.
39. Hripcsak G, Ludemann P, Pryor TA, Wigertz OB, Clayton PD. Rationale for the Arden Syntax. Comp and Biomed Res. 1994;27(4):291–324. [PubMed]
40. Wikipedia - The Free Encyclopedia Health Level 7 - Open Source Tools. Jun 11, 2007. [cited 2007 June]; Available from: http://en.wikipedia.org/wiki/HL7#Open_source_tools.
41. Kalra D. Electronic Health Record Standards. In: Kulikowski C, Haux R, editors. IMIA Yearbook of Medical Informatics 2006. Methods Inf Med. Vol. 45. 2006. pp. 136–44. Suppl 1.
42. Moner D, Maldonado JA, Bosca D, Fernandez JT, Angulo C, Crespo P, et al. Archetype-Based Semantic Integration and Standardization of Clinical Data. 28th Annual International Conference of the IEEE (EMBS '06): Engineering in Medicine and Biology Society. 2006:5141–4. [PubMed]
43. Smith S, Kaufman J, Jiang K, Davis M, Onvural M, Oprencak I. Eclipse Open Healthcare Framework; The Server-Side Eclipse Symposium at the 2007 European Eclipse Summit; Esslingen, Germany. 2007.
44. Introducing openEHR 2007 Jan, 2007. [cited 2007 April]; Available from: http://svn.openehr.org/specification/TRUNK/publishing/openEHR/introducing_openEHR.pdf.
45. Garde S, Knaup P, Hovenga E, Heard S. Towards Semantic Interoperability for Electronic Health Records: Domain Knowledge Governance for openEHR Archetypes. Methods Inf Med. 2007;46(1) [PubMed]
46. McCallum G. EGADSS: A Clinical Decision Support System for use in a Service-oriented Architecture. University of Victoria; 2006. [Master's Thesis]
47. de Clercq PA, Blom JA, Hasman A, Korsten HHM. Design and Implementation of a Framework to Support the Development of Clinical Guidelines. Int J Med Inform. 2001;64(2/3):285–318. [PubMed]
48. Gennari JH, Musen MA, Fergerson RW, Grosso WE, Crubézy M, Eriksson H, et al. The Evolution of Protégé: An Environment for Knowledge-based Systems Development. Int J Hum Comput Stud. 2003;58(1):89–123.
49. Chaudhri VK, Farquhar A, Fikes R, Karp PD, Rice JP. OKBC: a programmatic foundation for knowledge base interoperability; AAAI '98/IAAI '98: Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence; Menlo Park, CA, USA. 1998. pp. 600–7.
50. Musen MA. Domain Ontologies in Software Engineering: Use of Protégé with the EON Architecture. Methods Inf Med. 1998;37:540–50. [PubMed]
51. Tu SW, Musen MA. Modeling data and knowledge in the EON guideline architecture. In: Patel V, Rogers R, Haux R, editors. MEDINFO 2001: Proceedings of the 10th World Congress on Medical Informatics; Amsterdam, NL: IOS Press; 2001. pp. 280–84.
52. Boxwala AA, Peleg M, Tu SW, Ogunyemi O, Zeng Q, Wang D, et al. GLIF3: A Representation Format for Sharable Computer-interpretable Clinical Practice Guidelines. J Biomed Inform. 2004;37(3):147–61. [PubMed]
53. Skonetzki S, Gausepohl HJ, van der Haak M, Knaebel S, Linderkamp O, Wetter T. HELEN, a modular framework for representing and implementing clinical practice guidelines. Methods Inf Med. 2004;43(4):413–26. [PubMed]
54. Campbell JR, Tu SW, Mansfield JG, Boyer JI, McClay J, Parker C, et al. The SAGE Guideline Model: A Knowledge Representation Framework for Encoding Interoperable Clinical Practice Guidelines. In: Musen MA, editor. Proceedings of the 2003 AMIA Annual Symposium; Washington, DC. 2003.
55. Purves IN, Sugden B, Booth N, Sowerby M. The PRODIGY Project — the Iterative Development of the Release One Model. In: Lorenzi NM, editor. Proceedings of the 1999 AMIA Annual Symposium; Washington DC. 1999. pp. 359–63. [PMC free article] [PubMed]
56. Fox J, Johns N, Rahmanzadeh A. Disseminating Medical Knowledge: The PROforma Approach. Artif Intell Med. 1998;14:157–81. [PubMed]
57. Kosara R, Miksch S. Metaphors of Movement: A Visualization and User Interface for Time-Oriented, Skeletal Plans. Artificial Intelligence in Medicine. 2001;22(2):111–31. Special Issue: Information Visualization in Medicine. [PubMed]
58. Shahar Y, Miksch S, Johnson P. The Asgaard Project: A Task-Specific Framework for the Application and Critiquing of Time-Oriented Clinical Guidelines. Artif Intell Med. 1998;14:29–51. [PubMed]
59. Steele R, Fox J. Tallis PROforma Primer – Introduction to PROforma Language and Software with Worked Examples. Advanced Computation Laboratory, Cancer Research; London, UK: 2002.
60. Fox J, Das S. Safe and Sound: Artificial Intelligence in Hazardous Applications. AAAI Press; Menlo Park, CA: 2000. and MIT Press, Cambridge, Mass.
61. Sutton DR, Fox J. The Syntax and Semantics of the PROforma Guideline Modeling Language. J Am Med Inform Assoc. 2003;10(5):433–43. [PMC free article] [PubMed]
62. Advanced Computation Laboratory Tallis Composer User Interface. Sep 13, 2006. [cited 2007 May 20]; Available from: http://www.acl.icnet.uk/lab/tallis/Composer_User_Interface01.htm.
63. Polvani K-A, Agrawal A, Karras B, Deshpande A, Shiffman R. GEM Cutter Manual. New Haven, CT: 2000.
64. Shiffman RN, Karras BT, Agrawal A, Chen R, Marenco L, Nath S. GEM: A Proposal for a More Comprehensive Guideline Document Model Using XML. J Am Med Inform Assoc. 2000;7(5):488–98. [PMC free article] [PubMed]
65. Yale Center Medical Informatics GEM Cutter: Screenshot. [cited 2005 Oct. 20]; Available from: http://gem.med.yale.edu/GEM_Cutter/gem_cutter.htm.
66. Shiffman RN, Michel G, Essaihi A, Thornquist E. Bridging the Guideline Implementation Gap: A Systematic, Document-Centered Approach to Guideline Implementation. J Am Med Inform Assoc. 2004;11(5):418–26. [PMC free article] [PubMed]
67. Votruba P, Miksch S, Kosara R. Facilitating Knowledge Maintenance of Clinical Guidelines and Protocols. In: Fieschi M, Coiera E, Li Y-CJ, editors. MEDINFO 2004: Proceedings of the 11th World Congress on Medical Informatics; Amsterdam, NL: IOS Press; 2004. pp. 57–61.
68. Seyfang A, Miksch S, Marcos M, Wittenberg J, Polo-Conde C, Rosenbrand K. Bridging the Gap between Informal and Formal Guideline Representations. In: Brewka G, Coradeschi S, Perini A, Traverso P, editors. Proceedings of the 17th European Conference on Artificial Intelligence (ECAI-2006); Riva del Garda, Italy: IOS Press; 2006. pp. 447–51. Amsterdam, NL.
69. Kaiser K, Akkaya C, Miksch S. How Can Information Extraction Ease Formalizing Treatment Processes in Clinical Practice Guidelines? A Method and Its Evaluation. Artif Intell Med. 2007;39(2):151–63. [PMC free article] [PubMed]
70. Kaiser K, Miksch S. Modeling Treatment Processes Using Information Extraction. In: Yoshida H, Jain A, Ichalkaranje A, Jain LC, Ichalkaranje N, editors. Advanced Computational Intelligence Paradigms in Healthcare - 1. Springer Verlag; 2007. pp. 189–224.
71. Taboada M, Des J, MartÌnez D, Mira J. A Study of Applying Knowledge Modelling to Evidence-based Guidelines. In: ten Teije A, Lucas P, Miksch S, editors. Proceedings of the Workshop AI techniques in healthcare: evidence-based guidelines and protocols (at ECAI'06); 2006.
72. Ruzicka M, Svátek V. Mark-up Based Analysis of Narrative Guidelines with the Stepper Tool. In: Kaiser K, Miksch S, Tu SW, editors. Computer-based Support for Clinical Guidelines and Protocols. Proceedings of the Symposium on Computerized Guidelines and Protocols (CGP 2004); Amsterdam, NL: IOS Press; 2004. pp. 132–6.
73. Svátek V, Ruzicka M. Step-By-Step Mark-Up of Medical Guideline Documents. International Journal of Medical Informatics. 2003;70(2/3):319–35. [PubMed]
74. Shahar Y, Young O, Shalom E, Galperin M, Mayaffit A, Moskovitch R, et al. A framework for a distributed, hybrid, multiple-ontology clinical-guideline library, and automated guideline-support tools. J Biomed Inform. 2004;37(5):325–44. [PubMed]
75. Shalom E. Degel: Uruz Screenshot. 2006. [cited 2006 Sep 20]; http://medinfo.ise.bgu.ac.il/MedLab/ResearchProjects/RP_Uruz3.htm.
76. Musen MA. Preface: Protégé: community is everything. Int J Hum Comput Stud. 2005;62(5):545–52.
77. Jenders RA, Sailors RM. Convergence on a standard for representing clinical guidelines: work in health level seven. In: Fieschi M, Coiera E, Li YCJ, editors. MEDINFO 2004: Proceedings of the 11th World Congress on Medical Informatics; Amsterdam, NL: IOS Press; 2004. pp. 130–34.
78. Sanders GD. Automated Creation of Clinical Practice-Guidelines from Decision Models. Stanford University; 1998.
79. Zhu AL, Leong TY. Automating dynamic decision model construction to support clinical practical guideline development. In: Heller B, Loffler M, Musen M, Stefanelli M, editors. Computer-Based Support for Clinical Guidelines and Protocols: Proceedings of EWGLP 2000. IOS Press; 2001. pp. 133–148.
80. Serban R, ten Teije A, van Harmelen F, Marcos M, Polo-Conde C. Extraction and use of linguistic patterns for modelling medical guidelines. Artificial Intelligence in Medicine. 2007;39(2):137–49. [PubMed]
81. Zhu AL, Li J, Leong TY. Automated knowledge extraction for decision model construction: A data mining approach. In: Musen M, editor. Proceedings of the 2003 AMIA Annual Symposium; Washington, DC. 2003. pp. 758–62. [PMC free article] [PubMed]