Numerous factors influence whether and how guidelines are used. Some of these are extrinsic factors, such as: the nature of the newly recommended practice or technology itself; characteristics of healthcare providers; organizational capacity to collect, adapt, share, and apply evidence; and system-level environmental factors [8-10]. Because these manifest downstream of guideline development, single and combined interventions can be applied to address these barriers and improve compliance with guideline recommendations, although their impact can be variable and inconsistent [11]. Other factors are intrinsic to guidelines and perhaps best addressed at the time of guideline development. For example, guidelines may be biased through conflict of interest, of variable methodological quality, poorly written and ambiguously presented, viewed as not applicable to individual patients or reducing clinician autonomy, and the volume of guidelines now available may be overwhelming, particularly given that recommendations for the same clinical indication may be inconsistent across different guidelines [12-14].

To promote the development and use of high quality guidelines, the Appraisal of Guidelines Research and Evaluation (AGREE) instrument was developed by an international collaboration of guideline development experts [15]. A revised version, AGREE II, was recently issued [16]. It can be used to score guidelines based on their reported scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, editorial independence, and applicability. The Guideline Implementability Appraisal (GLIA) instrument is another tool that similarly considers the impact of guideline recommendations on individuals and organizations, and whether measurable indicators are provided by which to evaluate implementation [17]. Still, many guidelines fail to comply with these recommended standards. Shaneyfelt et al. evaluated 279 guidelines published from 1985 to June 1997 by 69 different developers, and found that mean overall adherence to standards for guideline format, methods, identification and summary of evidence, and formulation of recommendations was 43% [18]. When the AGREE instrument was used to evaluate 86 guidelines from 11 countries published between 1992 and 1999, they were found to vary considerably in whether and how they addressed each domain [19].

Turner et al. proposed that variability in manuals describing guideline development methods may be causing the mismatch between standards for guideline quality and the actual quality of guidelines [20]. They identified and examined six prominent guideline development manuals and found strong concordance in recommended guideline development methods across all manuals. However, the instructions varied in level of detail, so the authors hypothesized that manuals may not provide sufficient guidance on how to complete all steps of the guideline development process.

It is now well recognized that simply producing guidelines does not itself lead to use, and that active implementation is needed to encourage their uptake. The Knowledge to Action Cycle offers a framework for promoting the use of guidelines [21]. It recommends an iterative process comprised of several steps, including adapting guidelines to local context, identifying barriers of guideline use, selecting and implementing tailored interventions to promote guideline use, monitoring guideline use, evaluating outcomes associated with guideline use, and sustaining guideline use. Regardless of whether guideline developers are mandated and resourced to implement the guidelines they produce, or whether it is the responsibility of other individuals or organizations to implement guidelines, detailed instructions for guideline implementation are needed. Turner et al. found that four of six manuals described instructions for developing implementation advice that could be included in guidelines. However, the framework they used to examine manuals was based largely on the AGREE instrument, supplemented by a literature search for other key methods related to the development of guidelines. The framework consisted of 14 elements organized in four domains: preparing for guideline development, systematically reviewing the evidence, drafting the guideline, and reviewing the guideline. Only one element pertained to implementation, and they only looked for presence or absence of any implementation information, and not the details of instructions for developing implementation advice.

We investigated how to make guidelines more implementable by modifying their content and format [22]. We first conducted a review of the medical literature for features of guidelines desired by guideline users, or that are positively associated with guideline use. The framework was validated through review by health professionals and researchers with various clinical and disciplinary perspectives, and refined by using it to examine the content of published guidelines judged to be high quality by trained experts [22]. The final framework consisted of 22 elements within eight domains, including adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation (Table ​(Table1).1). Most guidelines we examined contained a large volume of graded evidence (validity) and numerous tables featuring complementary clinical information (applicability), but few contained additional features representing the other six domains that may facilitate application of guidelines by users. As proposed by Turner et al., guidelines may offer little support to users for implementing the recommendations because guideline development manuals lack instructions for developing such implementation advice. To confirm this and more thoroughly describe the nature of instructions for developing implementation advice in guideline development manuals, the purpose of this research was to re-examine the manuals sampled by Turner et al. according to our framework of implementability, which reflects guideline components shown by research to be associated with intent to, and actual use of guidelines. Use of a more detailed framework serves to pinpoint specific implementation information that may be lacking, thereby informing future updating of such manuals, or development of adjunct products.

Implementability domains and elements addressed in manuals are enumerated in Table ​Table3.3. Most manuals contained some representation of the majority of these domains and elements, though detail was minimal and did not lend itself to thematic analysis. COE included all domains but Applicability, but few elements within the domains of Relevance and Accommodation [25]. NHMRC [26] included all domains in the main manual or implementation supplement but few elements in the domain of Accommodation, as did NICE [27]. NZGG did not include the domains of Applicability and Communicability, and few elements in the domain of Accommodation [28]. SIGN did not include the domain of Applicability, and few elements in the domains of Accommodation and Implementation [29]. WHO did not include the domains of Adaptability and Comunicability, few elements in the domain of Accommodation, and cursory mention of Implementation and Evaluation [30].

While directions for creating Applicability information were absent in guideline development manuals, our previous work found that 90% of guidelines we examined did include clinical considerations by which to individualize recommendations [22]. This is likely because guidelines largely summarize data from clinical studies, and closely related to this is information such as diagnostic or risk criteria, pharmacologic dosing, indications for treatment or referral, and management options that often form the basis of Applicability content. The absence of Accomodation and Communicability information in guideline development manuals may not be surprising. Traditionally guidelines have often focused on questions relevant to clinicians, and when gaps in care emerged, the focus of quality improvement has been on changing clinician behaviour. Recognition of the need to consider a broader array of factors that influence guideline use and impact, and the role of guidelines as tools to direct quality improvement in this broader context is relatively recent [8-10,21]. Similarly, the role of patients in the patient-provider dyad, clinical decision making, and even guideline development is evolving [31]. In keeping with the fact that new paradigms are emerging, our previous work found that 50% of the guidelines we examined contained Communicability information, and less than 50% contained Accomodation information [22].

The guideline enterprise is challenged to keep up with these changes and modify the traditional guideline development and implementation strategy. This may require consideration of different knowledge sources and evidence, involvement of different types of experts in the guideline development and implementation process either in staged fashion or as multidisciplinary or interprofessional teams, and partnerships between those who synthesize and interpret clinical effectiveness data and those who plan and/or undertake implementation. We know that guideline implementation is complex, and many developers lack the mandate and/or resources to implement the guidelines they produce [32,33], so an overarching question that must be addressed is who bears responsibility for implementing guidelines, what resources are required to support implementation as conceived in this emerging, broader paradigm, and who will provide those resources.

Interpretation of these findings may be confounded by several issues. Not all existing guideline manuals were examined. We instead relied on the comprehensive search strategy applied by another research team to identify our sample. They chose to examine six manuals that were produced by major international guideline development organizations, and of a general nature and therefore of broad relevance. We did search for and examine newer versions of those manuals if they were available. We also ran an updated search for other general guideline development manuals, and found only one published by the Canadian Medical Association [34]. For consistency and comparative purposes, we examined only the six manuals that were included in the previous study but brief review of the CMA manual revealed that it consists of 34 pages, of which one eleven-page chapter on implementation partially addresses Implementation and a three-page chapter partially addresses Evaluation. Instructions reflecting the remaining implementability domains and elements is lacking, therefore this manual too provides guideline developers with little instruction for preparing implementation advice.

Our implementability framework, while formulated based on expressed needs among various types of guideline users and studies positively associating these features with actual guideline use, has not been thoroughly validated. We have yet to conduct experimental studies that would test whether inclusion of information or tools reflecting these implementability domains leads to guideline implementation, use, and beneficial healthcare outcomes. We also need to assess how different types of users (clinicians, managers, policy makers) would interpret and use implementability tools. First, however, we need to develop implementability tools because our most recent analysis of guidelines (not yet published) found that few contained such tools. In advance of such development, we are further validating the framework with input from international guideline developers, implementers, and researchers. Despite the need for further validation, our framework offers a more comprehensive way to evaluate implementation instructions in guideline development manuals that reflects the emerging, more complex multi-step paradigm of guideline implementation. Therefore, this study provides a more detailed evaluation than previously published on whether differences in intrinsic qualities of guidelines may be due to differences in the content of development manuals, and more precisely how manuals could be refined to facilitate the development of guidelines that provide users with implementation advice. These findings can be used by those who developed the manuals to consider expanding their content. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated.

Another limitation—not necessarily of our study, but in the evidence on guideline development and implementation—is the lack of criteria by which to assess the quality or actionability of instructions for developing guidelines or guideline implementation advice [35]. By using our more comprehensive implementability framework, we were able to comment on the completeness of the instructions. To judge the quality of the content of the instructions would require development of criteria, probably through expert consensus because little evidence is available, and then more detailed analysis of the content, both of which were beyond the resources available for the study described here. In ongoing research, we are searching for and examining the format and content of guidelines or adjunct products reflecting Accommodation, Implementation, and Evaluation to create ideal templates for tools offering implementation advice that will be vetted by both guideline developers and users.