We identified 26 unique guidelines from 1998–2001: 21 were retrieved and on assessment met our inclusion criteria [3
]. Five were excluded for the following reasons: three did not include clinical recommendations or were review articles [28
] and two were not retrievable [31
]. Consultation with content experts identified seven guidelines, results of a Medline search identified four further guidelines; search of the World Wide web yielded three guidelines, and known guideline websites yielded six potential guidelines and review of references a further seven guidelines. Eight of the 21 guidelines reported receiving outside funding, and 6/8 of the guidelines acknowledged funding from industry.
Twelve guidelines were published, seven were organizational reports, and two were accessible only from the web. Guideline developers included medical societies (n = 8), national groups (n = 6), international groups (n = 1), and government agencies (n = 6). Table presents the percentage of the guidelines that satisfied each of the 37 quality criteria. Figure illustrates the distribution of the guidelines by dimension quality score. Twenty of the 21 guidelines did not include patients/consumers on the development panel. Only 11 of the 21 included family physicians.
Frequency Distribution of Dimension Scores
Half or more of the 20 items assessing the rigor of guideline development were met by 15% of the guidelines (median quality score 23%, (95% CI 16.5, 34.7, range 5–80%), 81% met at least half of the 12 items assessing guideline content and context (median score 58%, (95% CI 50.8, 65.5), range 17–83%), and none met half or more of the items assessing guideline application (median score 0%, (95% CI -0.5, 12.6) range 0–47%).
Table presents the mean dimension quality scores for the 21 osteoporosis CPGs. Eight guidelines described the method used to interpret and assess the strength of evidence. Six of the 21 guidelines provided an explicit link between major recommendations and supporting evidence.
Mean Quality Scores of Osteoporosis Practice Guidelines (CPG), Maximum score = 100 for each Dimension (D)
The mean global quality score was 5.23 (95% CI 4.52, 5.94; median 5.33, standard deviation 1.56), suggesting that the appraisers felt the guidelines to be of medium quality.
The mean global assessment rating demonstrated that 11 of the 21 guidelines were recommended for use in practice (1 without change, 10 with modifications) and 10 guidelines were not recommended for use in practice.
Assessment of the reliability of the guideline appraisal instrument showed that D1 and D3 had good internal consistency, and D2 was borderline. The Cronbach's alpha was 0.87 for rigour of guideline development, 0.64 for context and content, and 0.71 for application.
For agreement between appraisers, the intraclass correlation coefficients were 0.91 for D1, 0.63 for D2 and 0.93 for D3. The appraisers' scores were within 20 percentage points for 95%, 67% and 100% of the guidelines for D1, 2, and 3, respectively. The Pearson's correlation coefficients between appraisers' dimension scores and their global quality scores (0–10) were 0.71 for D1, 0.60 for D2 and 0.46 for D3, with all coefficients being significant. The Pearson's correlation coefficients between appraisers' dimensions scores and their overall global assessment (recommend or not) were 0.59, 0.61 and 0.49 for D1, D2 and D3 respectively, with all coefficients being significant.
Eighteen of 21 guidelines included both pharmaceutical and non-pharmaceutical options (fall prevention, exercise or hip protectors). Ten of the guidelines included recommendations on fall prevention, and nine included recommendations on hip protectors.
There was no significant difference in the mean quality assessment when analyzed according to industry funded versus non-funded guidelines, type of publication (published, web or report), or the type of organization. There was no significant difference in quality according to year of publication, although there appeared to be a trend to improved quality from 1998 to 2001. Only 1/8 of the guidelines that acknowledged funding gave evidence that the potential biases were taken into account.