Many clinical guidelines – developed at substantial cost and effort – have proven to be difficult or impossible to operationalize. We developed the GuideLine Implementability Appraisal to facilitate guideline implementation. The instrument is designed to systematically highlight barriers to implementation.
The GLIA is intended to provide feedback about a guideline's implementability to two distinct audiences: the authors of the guideline and those individuals who choose guidelines for application within a health care delivery system. As a guideline is being developed, GLIA can provide feedback to guideline authors about potentially remediable defects. Developers may choose to make modifications to the guideline document before it is finalized and disseminated. Implementers can use GLIA to help select a guideline, to identify potential obstacles, and to target efforts toward addressing identified barriers. Thus, GLIA can be used to help select guidelines that are more easily implementable and also to devise implementation strategies that address identified barriers.
Two GLIA dimensions are of particular importance because failure to address them adequately will result in inconsistent implementation [23
]. Any recommendation that does not clearly communicate what to do (i.e., it fails executability criteria) or when to do it (i.e., fails decidability criteria) is not fully ready for implementation. If possible, guideline authors should revise such a recommendation before it is disseminated for implementation. If problems in decidability and executability are not corrected prior to dissemination, different implementers may well interpret the guideline authors' intent in a discordant manner.
GLIA incorporates an optional dimension – computability – to indicate the ease with which a recommendation might be operationalized in an electronic information system. Guideline implementation strategies – e.g., education, academic detailing, audit and feedback, administrative sanctions – need not necessarily involve computers [25
]. Because of the success of electronic implementations in influencing clinician behavior [26
], the wide variation in electronic information systems, and the current lack of guidance regarding computability, these extrinsic considerations were retained in the instrument. Items in the computability
dimension address the availability of data to trigger the recommendation, the level of specificity of the triggers and recommended actions, and whether there is a clear path from recommendation to electronic implementation.
Implementability must be differentiated from guideline quality. Quality assessments relate primarily to determining the scientific validity of guidelines and, generally, quality is assessed for the guideline as a whole. Implementability, on the other hand, is one component of guideline quality, but its assessment is applied largely to individual recommendations within a guideline.
In a comprehensive review of 13 tools for guideline quality appraisal, Graham [29
] identified instruments developed by Cluzeau [14
] and Shaneyfelt et al [16
] as the best developed. Since that time, the AGREE Instrument [15
] has emerged as the leading exemplar of guideline quality appraisal and it has been endorsed by the Guidelines International Network [30
]. Application of GLIA can complement quality appraisal in identifying guideline deficiencies. Several GLIA items overlap with items in the AGREE scale. However, to the best of our knowledge, GLIA is the only tool that emphasizes implementation concerns at the level of the individual recommendation.
Application of GLIA to measurement of decidability and executability requires that users translate guideline recommendations into statements comprising conditions and actions [31
]. Training and practice may be required to assure consistency of logical analysis.
Application of GLIA requires team effort and a consequent resource investment. The team that applies GLIA should include members with skills in guideline implementation as well as members with specific understanding of the clinical domain. Our experience has demonstrated that rating a guideline that contains 15 recommendations might require several hours of an individual's time. Additional time must be spent in resolving divergent ratings, although this effort usually yields an improved understanding of implementation issues. We are currently developing an electronic version of GLIA that will provide a more efficient means of rating, scoring, and reporting results.
During development, GLIA is best applied once evidence has been synthesized and draft recommendations have been formulated. When applied too late in the authoring process, GLIA may have limited impact because authors may have already become committed to recommendations as written and thus not open to making modifications. Assuring that guideline authors understand the importance of implementability early on may help to overcome premature commitment.
The classic challenge of instrument development is to arrive at the correct number of items to minimize burden and avoid redundancy, while including a sufficient number to be comprehensive. GLIA contains 7 global items that are applied once to each guideline, 20 items that are applied to each rated recommendation, and 4 optional items rating computability that are applicable when an electronic implementation is planned. Further use of GLIA is likely to result in clarification and perhaps modification of the number of items.
We have performed a series of activities to provide preliminary evidence of GLIA's validity. However, neither the inter-rater reliability of GLIA, the test-retest reliability, the factor structure of the dimensions, nor its predictive validity has yet been established. Plans for this testing are underway.
It should be noted that the authors performed the activities described as Step 4. Until verified by an independent panel, these results should be considered preliminary and subject to potential bias.
In addition, GLIA addresses primarily factors intrinsic to the guideline. It is clear that extrinsic factors are critical in a successful implementation. Future extensions to GLIA will help to identify extrinsic barriers to implementation.