Attributes of clinical guidelines that influence use of guidelines in general practice: observational study

BMJ. 1998 September 26; 317(7162): 858–861.

PMCID: PMC31096

Attributes of clinical guidelines that influence use of guidelines in general practice: observational study

Richard Grol, professor,^a Johannes Dalhuijsen, general practitioner,^b Siep Thomas, director of guidelines development,^b Cees in ’t Veld, general practitioner,^b Guy Rutten, associate professor,^c and Henk Mokkink, senior researcher^a

^aCentre for Quality of Care Research, Universities of Nijmegen-Maastricht, PO Box 9101, 6500 HB Nijmegen, Netherlands, ^bDutch College of General Practitioners, PO Box 3231, 3502 GE Utrecht, Netherlands, ^cDepartment of General Practice, Utrecht University, PO Box 80045, 3508 TA Utrecht

Contributed by

Contributors: RG coordinated the study, had the original idea, performed the literature search, and helped analyse the data. JD contributed to the literature search, collected data in the practices, and contributed to the data analysis. JD, ST, CV, and GR developed the attributes instrument and assessed the recommendations. HM participated in the coordination of the study and performed most of the analyses. The paper was written jointly by RG, JD, and HM, and the other authors revised the various drafts of the manuscript.

Correspondence to: Professor Grol rgrol/at/hsu.kun.nl

Accepted August 27, 1998.

This article has been cited by other articles in PMC.

Abstract

Objective: To determine which attributes of clinical practice guidelines influence the use of guidelines in decision making in clinical practice.

Design: Observational study relating the use of 47 different recommendations from 10 national clinical guidelines to 12 different attributes of clinical guidelines—for example, evidence based, controversial, concrete.

Setting: General practice in the Netherlands.

Subjects: 61 general practitioners who made 12

880 decisions in their contacts with patients.

Main outcome measures: Compliance of decisions with clinical guidelines according to the attribute of the guideline.

Results: Recommendations were followed in, on average, 61% (7915/12

880) of the decisions. Controversial recommendations were followed in 35% (886/2497) of decisions and non-controversial recommendations in 68% (7029/10

383) of decisions. Vague and non-specific recommendations were followed in 36% (826/2280) of decisions and clear recommendations in 67% (7089/10

600) of decisions. Recommendations that demanded a change in existing practice routines were followed in 44% (1278/2912) of decisions and those that did not in 67% (6637/9968) of decisions. Evidence based recommendations were used more than recommendations for practice that were not based on research evidence (71% (2745/3841) v 57% (5170/9039)).

Conclusions: People and organisations setting evidence based clinical practice guidelines should take into account some of the other important attributes of effective recommendations for clinical practice.

Key messages

Specific attributes of clinical practice guidelines determine whether they are used in practice
Evidence based recommendations are better followed in practice than recommendations not based on scientific evidence
Precise definitions of recommended performance improve the use of guidelines
Testing the feasibility and acceptance of clinical guidelines among the target group is important for effective implementation
People setting evidence based guidelines need to understand the attributes of effective guidelines

Introduction

The increase in the number of clinical guidelines produced and published in different countries has stimulated discussion on their value. How good are they? Are they based on the best scientific evidence available? How effective are they in normal clinical practice? Interest has developed in many countries in the attributes of successful clinical guidelines.¹^–⁵ The scientific validity and reliability of the guidelines receive most attention, less attention being paid to the features of guidelines that may determine their use in decision making in clinical practice. To date, research on these attributes has also been scarce.⁶ A good understanding of which attributes of guidelines influence their use in daily clinical practice is crucial for guideline development to be cost effective. Developing guidelines is laborious and usually expensive. Regrettably, too many guidelines do not remain in regular use, although the aim is to implement them in clinical practice. We therefore determined which attributes of guidelines or specific recommendations for appropriate care are related to their use in practice and in clinical decision making.

Subjects and methods

Determination of attributes of guidelines

We conducted a non-systematic study of scientific articles and documents from professional and scientific organisations in different countries on guideline development and attributes of guidelines to select attributes that might influence the use of guidelines in practice.¹^–²⁶ Attributes were concerned with the scientific validity of guidelines; their relevance and applicability in practice; the formulation and style of the recommendations; their compatibility with existing opinions and values; their complexity; their consequences for care providers, patients, doctors, and practice management; the risks of applying the recommendations; and the attention given to the guideline in the dissemination process. We formulated a set of 16 different attributes on the basis of these documents (box).

Attributes of guidelines

(1) The recommendation is based on scientific evidence—an explicit description of the scientific evidence for the recommendation is available; the research evidence is straightforward and not conflicting; the recommendation is based on the results of well designed clinical trials or meta-analyses

(2) The recommendation is based on clear and convincing arguments that are based on extensive clinical skills and experience

(3) The recommendation is concerned with a relevant aspect of care in daily practice

(4) The recommendation helps doctors to solve patients’ problems in daily care—it is concerned with difficult decisions or choices in daily care and it makes work easier

(5) The recommendation is one of the key features of the guideline—it is a central element in the guideline and represents the central aim

(6) The recommendation provides a concrete and precise description of desired performance—it gives detailed advice on which performance is appropriate in which situation and in what patient group and determines which factors or conditions should be taken into account

(7) The recommendation is vague and not specific

(8) The recommendation is complex—it is composed of many different elements and contains a complex decision tree or many different conditional factors influencing performance

(9) The recommendation is not compatible with existing norms and values in practice—it is controversial and provokes discussion

(10) The recommendation demands the acquisition of new competence (knowledge, skills)—it can be followed only when a doctor has specific knowledge and skills

(11) The recommendation has specific consequences for practice management—it requires adaptations in the organisation of care processes or demands extra resources, staff, equipment, etc

(12) The recommendation demands changing existing routines and habits and leaving what is seen as common practice in the target group

(13) The recommendation will provoke negative reactions in patients because it does not fit their common expectations—it may lead to a conflict of interest between patient and doctor

(14) The recommendation will provoke negative reactions among colleagues because it is not compatible with their views, position, or tasks

(15) The recommendation can be tried without any risks of possible damage for patients—experimenting with the proposed performance will not have negative effects on the health of patients

(16) The recommendation has been mentioned in the media and in implementation programmes

Attributes of guidelines in Dutch general practice

We selected 47 recommendations from 10 different national guidelines for general practice developed by the Dutch College of General Practitioners.²⁷^,28 These national guidelines are developed rigorously by combining the results of a systematic analysis of the scientific evidence with the results of consensus discussions of working parties of experienced general practitioners and specialists. These guidelines are published in the scientific journal for general practitioners Huisarts en Wetenschap [GP and Science] and disseminated through specially developed programmes of continuing medical education. Most general practitioners are knowledgeable about the guidelines 6-12 months after publication.²⁷ Each guideline consists of a set of 20-50 recommendations. In this study we selected a few recommendations on the management of acne vulgaris, acute sore throat, ankle distortion, acute otitis media, shoulder complaints, urinary tract infection, diagnostics in ophthalmology, type 2 diabetes, hypertension, and the prescription of oral contraceptives. Eight of the recommendations were concerned with management of chronic diseases and 39 with the management of acute problems in general practice. Twenty six focused on diagnostic performance, eight on therapeutic performance, six on patient education and advice, and seven on follow up or referral to specialists.

Four of us (JD, ST, CV, GR) independently determined whether the attributes given in the box were present, partly present, or not present in the 47 recommendations. Agreement between the four of us was moderate (kappa averaged 0.46 over 6×16×47=4512 paired assessments), so we decided that three of us had to agree on the score of a recommendation to determine the final score (this was the case in 87% of the assessments). The remaining scores were decided by consensus in a structured meeting. Attribute 2 in the box was excluded because we interpreted it in different ways.

Compliance with Dutch guidelines in relation to attributes

We related the attribute scores to use of the recommendations in clinical decisions using data on the 47 recommendations collected as part of a large clinical audit among 61 general practitioners in the Netherlands.²⁷ The general practitioners were selected on the basis of their interest in the project, and they were comparable to the national population of general practitioners in type and location of practice and their attitude towards the national guidelines but they were somewhat younger. Over three months they recorded their performance on specially designed forms after each consultation in which one of the 10 national guidelines was applicable. A test on the reliability of this self recording procedure was performed by comparing the scores of four general practitioners working in normal, busy practice settings with the scores of an independent trained observer in the surgery. These comparisons showed that the general practitioners completed the forms reliably (kappa averaged 0.76 for different decisions).

Analysis

The opinions of the assessors on the presence of attributes were dichotomised ((partly) present v not present) for each of the 47 recommendations. Attributes that did not discriminate—that is, were present or not present in less than four of the recommendations—were not included in further analyses. This was the case for attributes 3, 14, and 15 in the box. Thus a final set of 12 attributes was used in the analyses (see table table1).1). The presence of these attributes in the 47 recommendations was established. Compliance with these recommendations in decision making in clinical practice was determined by distinguishing between the recommendations with an attribute present and those without an attribute present. For example, in 3841 cases or decisions made by general practitioners it was possible to determine whether the recommendations of 15 evidence based guidelines had been followed, and in 9039 cases or decisions made by general practitioners it was possible to determine whether the recommendations of 32 non-evidence based guidelines had been followed (see table table1).1). Thus for each decision made by the 61 general practitioners performance in practice could be compared with a specific recommendation. We evaluated 12

880 different decisions of general practitioners in this way to determine compliance.

Table 1

Compliance with 47 recommendations with different attributes of 12

880 decisions made by 61 general practitioners

Differences in compliance rates for the different attributes were tested using analysis of variance. To evaluate which attributes had most influence a stepwise regression analysis was performed using the total compliance rate for 12

880 decisions as the dependent variable.

Results

The recommendations were followed in, on average, 61% (7915/12

880) of the decisions of general practitioners. All 12 selected attributes influenced the use of the recommendations in practice, but the degree of influence was different for each attribute (table (table1).1). Whether a recommendation was controversial and incompatible with existing values in the target group and whether it was clearly defined were particularly important. Also its effect on daily work (attributes 10, 12, and 13; table table1)1) influenced use in clinical practice. Recommendations based on evidence were used more than those that were not.

The relative contribution of the different attributes to the total compliance rate for the 47 recommendations determined by stepwise regression analysis showed that the three attributes with the highest single correlation contributed most to the explanation of the variance (table (table2).2). These were the recommendation is controversial and not compatible with current values; the recommendation is vague and not precisely defined; and the recommendation demands change of fixed routines. These attributes mainly had an independent effect on the compliance rate in practice. They explained 17% of the total variance.

Table 2

Attributes determining compliance with recommendations in stepwise multiple regression analysis, with total score for compliance as dependent variable

Discussion

Scientists and policy makers need to understand the features of guidelines that relate to implementation of guidelines in decision making in daily practice.²⁹ Our study shows some of the important attributes of effective recommendations for clinical practice. Guidelines should be compatible with existing values among the target group and not be too controversial. They should not demand too much change to existing routines and be defined precisely, with specific advice on actions and decisions in different cases. They should be compatible with current values and routines. Indeed, some recommendations probably expressed what general practitioners were already prepared to do. The scientific basis of the recommendation was also important. Recommendations were more adhered to when an explicit description of the scientific evidence was available and the evidence was straightforward and not conflicting. The perceived consequences for doctors and practice management matter. A recommendation was used less when compliance affected the organisation of and staff in practices, when it demanded extra resources or acquisition of new knowledge and skills, or when it provoked negative reactions in patients.

The explained variance was not high (17%), but many other factors need to be taken into consideration to determine whether a guideline will be used. Setting guidelines is only one step in a comprehensive process of implementation and making patient care more effective.⁴^,7 Further research on the implementation of guidelines needs to provide insight into why some guidelines are effective and others are not. Our findings are consistent with theoretical models on effective guidelines.²⁹ To our knowledge, this is the first time in one study that the influence of different attributes of guidelines has been shown using empirical data from clinical practice. Our findings show that developers of guidelines need to take these attributes into account. They show the importance not only of performing an analysis of the scientific literature but also of carrying out a test in clinical practice to evaluate the feasibility and effectiveness of guidelines in daily care. They show the importance of a precise definition of the recommended performance in clinical decisions. Professional writers might be helpful in analysing the drafts of guidelines for vague and ambivalent use of language.

We used only a limited number of recommendations. Our selection may have been too limited and the whole range of clinical decisions may have to be sampled. We did, however, include recommendations for acute and chronic diseases and on diagnosis, treatment, advice, and follow up. We included over 12

000 decisions in clinical practice. The assessment of the recommendations by a panel of general practitioners could be improved as the reliability of its assessments was moderate. On the other hand, four general practitioners were able to reach consensus after discussing each other’s arguments. Our results do not allow conclusions to be drawn about causal relations, for which prospective, controlled trials comparing the effects of different types of recommendations are needed. People who draw up guidelines or finance the development of guidelines may improve the effectiveness of their work by evaluating their products with the attributes we have described.

Footnotes

Funding: The study was funded by grants from the Dutch Ministry of Health and the Health Insurance Council.

Conflict of interest: None.

References

1. Grimshaw J, Russel I. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317–1322. [PubMed]

2. Field M, Lohr K. Guidelines for clinical practice: from development to use. Washington, DC: National Academy Press; 1992.

3. Grimshaw J, Russel I. Achieving health gain through clinical guidelines. I. Developing scientifically valid guidelines. Quality in Health Care. 1993;2:243–248. [PMC free article] [PubMed]

4. Grimshaw J, Freemantle N, Wallace S, Russel I, Hurwitz B, Watt I, et al. Developing and implementing clinical guidelines. Quality in Health Care. 1995;4:53–64.

5. Royal College of General Practitioners Clinical Guidelines Working Group. The development and implementation of clinical guidelines. London: RCGP; 1995.

6. Grilli R, Lomas J. Evaluating the message: the relationship between compliance rate and the subject of a practice guideline. Med Care. 1994;32:202–213. [PubMed]

7. Grol R. Implementing guidelines in general practice care. Quality in Health Care. 1992;1:184–191. [PMC free article] [PubMed]

8. Kanouse D, Kallich J, Kahan J. Dissemination of effectiveness and outcome research. Health Policy. 1995;34:167–192. [PubMed]

9. Haines A, Jones R. Implementing findings of research. BMJ. 1994;308:1488–1492. [PMC free article] [PubMed]

10. Rogers E. Diffusion of innovations. New York: Free Press; 1983.

11. Cluzeau F, Littlejohns P, Grimshaw J, Hopkins A. Appraising clinical guidelines and development of criteria—a pilot study. J Interprofessional Care. 1995;9:227–229.

12. Woolf S. Practice guidelines: a new reality in medicine. II. Methods of developing guidelines. Arch Intern Med. 1992;152:946–952. [PubMed]

13. Hadorn D, Baker D. Development of the AHCPR-sponsored heart failure guideline: methodologic and procedural issues. Joint Commission Journal of Quality Improvement. 1994;20:539–547. [PubMed]

14. Eccles M, Clapp Z, Grimshaw J, Adams P, Higgins B, Purves I, et al. Developing valid guidelines: methodological and procedural issues from the North of England Evidence Based Guideline Developing Project. Quality in Health Care. 1996;5:44–50. [PMC free article] [PubMed]

15. Field MJ, editor. Setting priorities for clinical practice guidelines. Washington, DC: National Academy Press; 1995.

16. Lohr K. The quality of practice guidelines and the quality of health care. In: Selbmann H-K, editor. Guidelines in health care. Baden-Baden: Nomos; 1998.

17. National Health and Medical Research Council Australia. Guidelines for the development and implementation of clinical practice guidelines. Canberra: Australian Government Publishing Service; 1995.

18. Grol R. Development of guidelines for general practice care. Br J Gen Pract. 1993;43:146–151. [PMC free article] [PubMed]

19. American Medical Association. Attributes to guide the development of practice parameters. Chicago: AMA; 1994.

20. Hayward R, Laupacis A. Initiating, conducting and maintaining guidelines development programs. Can Med Assoc J. 1993;148:507–512. [PMC free article] [PubMed]

21. Eddy D. A manual for assessing health practices and designing practice policies. Philadelphia: American College of Physicians; 1992.

22. Duff L, Kitson A, Seers K, Humphris D. Clinical guidelines: an introduction to their development and implementation. J Advanced Nursing. 1996;23:887–895. [PubMed]

23. Winkler J, Lohr K, Brook R. Persuasive communication and medical technology assessment. Arch Intern Med. 1985;145:314–317. [PubMed]

24. Kanouse D, Jacoby J. When does information change practitioners’ behaviour? Int J Technol Assess Health Care. 1988;4:27–33. [PubMed]

25. Kahan J, Kanouse D, Winkler J. Stylistic variations in NIH Consensus statements 1979-1983. Int J Technol Assess Health Care. 1988;4:289–304. [PubMed]

26. Leape L. Practice guidelines and standards. Qual Rev Bull. 1990;17:42–49. [PubMed]

27. Grol R, Thomas S, Roberts R. Development and implementation of guidelines for family practice: lessons from the Netherlands. J Fam Pract. 1995;40:435–439. [PubMed]

28. Thomas S. Standard setting in the Netherlands: impact of the human factor on guideline development. Br J Gen Pract. 1994;44:242–243. [PMC free article] [PubMed]

29. Grol R. Beliefs and evidence in changing clinical practice. BMJ. 1997;315:418–421. [PMC free article] [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Publishing Group