|Home | About | Journals | Submit | Contact Us | Français|
Familiarity with guidelines is generally thought to be associated with guideline implementation, adherence and improved quality of care. We sought to determine if self-reported familiarity with acute respiratory infection (ARI) antibiotic treatment guidelines was associated with reduced or more appropriate antibiotic prescribing for ARIs in primary care.
We surveyed primary care clinicians about their familiarity with ARI antibiotic treatment guidelines and linked responses to administrative diagnostic and prescribing data for non-pneumonia ARI visits.
Sixty-five percent of clinicians responded to the survey question about guideline familiarity. There were 208 survey respondents who had ARI patient visits during the study period. Respondents reported being ‘not at all’ (7%), ‘somewhat’ (30%), ‘moderately’ (45%) or ‘extremely’ (18%) familiar with the guidelines. After dichotomizing responses, compared with clinicians who reported being less familiar with the guidelines, clinicians who reported being more familiar with the guidelines had higher rates of antibiotic prescribing for all ARIs combined (46% versus 38%; n = 11 164; P < 0.0001), for antibiotic-appropriate diagnoses (69% versus 59%; n = 3213; P < 0.0001) and for non-antibiotic appropriate diagnoses (38% versus 28%; n = 7951; P < 0.0001). After adjusting for potential confounders, self-reported guideline familiarity was an independent predictor of increased antibiotic prescribing (odds ratio, 1.36; 95% confidence interval, 1.25–1.48).
Self-reported familiarity with an ARI antibiotic treatment guideline was, seemingly paradoxically, associated with increased antibiotic prescribing. Self-reported familiarity with guidelines should not be assumed to be associated with consistent guideline adherence or higher quality of care.
Familiarity with guidelines is generally thought to be associated with guideline adherence and improved quality of care. Many studies use guideline familiarity or knowledge as outcomes to assess the effectiveness of educational interventions or as measures of the diffusion of guidelines. In primary care alone, investigators have surveyed clinicians’ familiarity or knowledge of guidelines for lead screening , otitis media , asthma [3–7], cardiovascular disease [8–10], hypertension , tobacco treatment , chronic kidney disease , hepatitis C  and colorectal cancer screening [15, 16], among others.
Acute respiratory infections (ARIs)—including non-specific upper respiratory infections, otitis media, sinusitis, pharyngitis, acute bronchitis, pneumonia and influenza—are the most common symptomatic reason for seeking ambulatory care in the USA, accounting for approximately 7% of all visits . ARIs are also the number one reason for antibiotic prescribing in the USA, accounting for about 50% of antibiotic prescriptions to adults . Much antibiotic prescribing for ARIs is inappropriate due to clinicians prescribing antibiotics for viral conditions or prescribing unnecessarily broad-spectrum antibiotics when a narrower-spectrum antibiotic would suffice [19–22].
In 2001, the American College of Physicians (ACP), other specialty societies and the Centers for Disease Control and Prevention (CDC), released the ‘Principles of Appropriate Antibiotic Use for Acute Respiratory Tract Infections in Adults’ . These guidelines aim to reduce antibiotic prescribing overall and reduce inappropriate antibiotic prescribing for predominantly viral infections, including non-specific upper respiratory tract infections, sinusitis, pharyngitis and acute bronchitis [24–27].
We hypothesized that familiarity with these guidelines would be associated with consistent adherence and decreased antibiotic prescribing, especially for diagnoses for which antibiotics are inappropriate. We surveyed clinicians and linked their responses to visit and prescribing data to determine if self-reported familiarity with the guidelines was associated with more appropriate antibiotic prescribing for ARIs in primary care.
Partners HealthCare System is an integrated regional health delivery network in eastern Massachusetts. The Partners Primary Care Practice-Based Research Network includes 27 primary care clinics that use the electronic Longitudinal Medical Record, or LMR, as their official medical record. Data for this evaluation were collected as part of a larger study of an LMR-based clinical decision support system aimed at improving the care of patients with coronary artery disease, diabetes and ARIs [28, 29].
These 27 clinics have approximately 200 staff physicians. In addition, about 20 non-physicians provide primary care and approximately 300 residents have continuity practices. The Partners HealthCare Institutional Review Board approved the study protocol.
We surveyed primary care clinicians, including staff physicians, non-physician clinicians and residents, twice between December 2005 and August 2006 to primarily gauge their attitudes and use of the decision support system. Potential respondents were notified of the surveys via email, which included a link to the online survey. We sent up to three reminders to complete the surveys.
The first survey, administered between December 2005 and June 2006, had 84 individual items, including questions on demographic information, clinical sessions, electronic health record (EHR) usage, workflow, attitudes towards decision support and satisfaction with and barriers to ARI and chronic disease management. The survey included a question that asked ‘how familiar are you with the following guideline: The ACP/CDC Principles of Appropriate Antibiotic Use for Acute Respiratory Tract Infections in Adults?’ Responses were ‘not at all,’ ‘somewhat,’ ‘moderately’ or ‘extremely.’ The second survey, administered between June 2006 and August 2006, had 59 individual items and included questions about electronic reminders, as well as the same question on familiarity with the ACP/CDC guidelines.
We linked survey responses to administrative diagnostic data and electronic prescribing data, limiting the analysis to respondents who saw at least one patient with a visit for an ARI diagnosis between November 2005 and May 2006 (the intervention period). We identified ARI visits using the International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM). We included ARI visits with ICD-9-CM codes for non-specific upper respiratory infections (ICD-9-CM 460, 464 and 465), otitis media (ICD-9-CM 381 and 382), sinusitis (ICD-9-CM 461 and 473), streptococcal pharyngitis (ICD-9-CM 034.0), non-streptococcal pharyngitis (ICD-9-CM 462 and 463), acute bronchitis (ICD-9-CM 466 and 490) and influenza (ICD-9-CM 487). We excluded visits with an ICD-9-CM code of pneumonia (ICD-9-CM 481–486; n = 628) as the ACP/CDC guidelines do not refer to pneumonia. We considered otitis media, sinusitis and streptococcal pharyngitis antibiotic-appropriate diagnoses. We considered non-specific upper respiratory tract infections, non-streptococcal pharyngitis, influenza and acute bronchitis non-antibiotic-appropriate diagnoses. We previously found that ICD-9-CM codes for ARIs, compared with clinician-assigned, chart-documented diagnoses, had a sensitivity of 98%, specificity of 96% and positive predictive value of 96% .
We combined clinicians who reported that they were ‘not at all familiar’ or ‘somewhat familiar’ and compared them to clinicians who reported being ‘moderately familiar’ or ‘extremely familiar’ with the guidelines. For clinicians who completed both the first and the second surveys, we used their responses from the first survey. Secondary analyses using only respondents from the first survey or excluding clinicians who ‘crossed over’ from the first to the second survey (i.e. from less to more familiar or more to less familiar) did not change the results substantively.
The primary outcome of interest was the antibiotic prescribing rate for ARI visits. We identified antibiotic prescribing through electronic prescribing within the EHR. We previously reported that EHR-based antibiotic prescribing was increasing rapidly over time, from 22% of prescriptions in 2000 to 58% in 2003 , and, during the study period, it was the policy of the study clinics that all prescriptions be written using the EHR. We considered antibiotic use the prescription of an orally administered antibiotic agent within 3 days after an ARI visit. Secondary outcomes included antibiotic prescribing for antibiotic-appropriate diagnoses, antibiotic prescribing for non-antibiotic-appropriate diagnoses and antibiotic prescribing for individual ARIs.
We examined characteristics of clinicians, including clinician age, gender and type of clinician (staff physician; fellow; resident; nurse practitioner or physician assistant; or other). Using registration data, we examined patients’ age, gender, race/ethnicity, primary language, primary insurance and median income by zip code. We also categorized clinic type as hospital-based, community health center or community-based.
The primary unit of analysis was the visit. We used standard descriptive statistics, comparing categorical variables using the chi-squared test and continuous variables using Student's t-test. To assess whether there was a significant trend in antibiotic prescribing across categories of guideline familiarity, we used the Cochran-Armitage trend test. To assess the independent association between self-reported guideline familiarity and antibiotic prescribing, we entered all available covariates into a multivariate logistic regression model with antibiotic prescribing as the independent variable. We modeled patient age, clinician visits per week and mean income by zip code as continuous predictors. We also adjusted for clustering by clinician using PROC GENMOD. We used SAS version 9.1 (SAS Institute, Inc, Cary, NC) for all analyses. We considered P values < 0.05 significant.
We emailed a link to the survey to 441 primary care clinicians. There were 326 clinicians (74%) who answered at least one survey question and 287 clinicians (65%) who answered the question about familiarity with the ACP/CDC guidelines on either the first or the second survey. There were 208 (72%) survey respondents who had a claims-derived diagnosis for an ARI between November 2005 and May 2006 and had responded to the guideline familiarity question.
Of these 208 clinicians, 74 answered the question about familiarity with the ACP/CDC guidelines on the first survey only, 31 on the second survey only and 103 on both the first and second surveys. From the first to the second survey, 10 clinicians crossed over from reporting they were less to more familiar with the ACP/CDC guidelines and 23 clinicians crossed over from reporting they were more to less familiar with the ACP/CDC guidelines.
The 208 clinicians who responded to the survey and had an ARI visit had a mean age of 39 years, 63% were female, 53% were staff physicians and 33% were residents (Table 1). Respondents who reported being more familiar with the guideline were more likely to be staff physicians and to report seeing more patients per week.
Visits to clinicians who reported being more familiar with guidelines were more likely to be made by younger patients, women, Latinos, Spanish-speaking patients and patients with health maintenance organization insurance (Table 2). Mean income by zip code was also lower for patients making visits to clinicians who reported greater familiarity with the guidelines. Patients at visits in which antibiotics were prescribed were more often white, English-speaking and less often had Medicare or self-pay insurance than patients at visits in which antibiotics were not prescribed.
Clinicians prescribed antibiotics in 44% of 11 164 non-pneumonia ARI visits (Table 3). Clinicians who reported being ‘not at all’ (n = 15), ‘somewhat’ (n = 62), ‘moderately’ (n = 93) and ‘extremely’ (n = 38) familiar with the guidelines prescribed antibiotics in 42, 37, 46 and 46% of visits, respectively (P for trend <0.0001). After dichotomizing the responses to guideline familiarity, clinicians who reported being more familiar with the guidelines prescribed antibiotics in 46% of visits and clinicians who reported being less familiar with the guidelines prescribed antibiotics in 38% of visits [odds ratio (OR), 1.40; 95% unadjusted confidence interval (CI), 1.30–1.52; P < 0.0001]. After adjusting for clustering by clinician, self-reported familiarity with the guidelines remained marginally statistically significant (OR, 1.3; 95% CI, 1.0–1.7; P = 0.04). Clinicians who reported greater familiarity with the ACP/CDC guidelines prescribed antibiotics more commonly for antibiotic-appropriate diagnoses and antibiotic-inappropriate diagnoses, as well as most individual ARIs.
In multivariable modeling, after adjusting for all available clinician, clinic and patient factors, self-reported familiarity with the ACP/CDC guidelines was still positively associated with antibiotic prescribing for ARIs (OR, 1.36; 95% CI, 1.25–1.48; Table 4). Female clinicians, older clinicians and residents prescribed antibiotics less frequently. Nurse practitioners and physician assistants and busier clinicians prescribed antibiotics more frequently. Clinicians prescribed antibiotics less frequently at community-based clinics, community health centers and to patients who were Black, were of other race/ethnicity or who were self-pay. After adjusting for clustering by clinician the odds of antibiotic prescribing by clinicians who reported being more familiar with the guideline was 1.3 (95% CI, 1.0–1.7; P = 0.03). The results did not change substantively when examining guideline familiarity in four categories or when adjusting for date-of-visit.
Guidelines are developed and disseminated in the hopes that their implementation will lead to improved quality of care. However, adoption of and adherence to practice guidelines is complex  and there are many steps between guideline dissemination and guideline adherence in clinical practice. Clinician awareness of a guideline is a necessary, but far from sufficient, step in this process. The ‘awareness-to-adherence’ model and similar models state that clinicians must be aware of a guideline, agree with it, adopt it as a part of care and regularly follow guideline recommendations [11, 32, 33]. Cabana et al.  developed a more detailed framework—including awareness of guidelines and guideline familiarity—describing why clinicians may not adhere to clinical practice guidelines. Seemingly paradoxically, we found that self-reported familiarity with the ACP/CDC guidelines for the antibiotic treatment of ARIs was related to increased antibiotic prescribing rates, despite the fact that these guidelines discourage antibiotic use for many ARI diagnoses.
However, this apparent paradox may be the result of findings described in the psychological literature. It is generally well known that people overestimate their performance . More recently understood, to a degree that cannot be explained by regression to the mean, people who perform well on a task only slightly underestimate their performance, whereas poor performers consistently and greatly overestimate their own performance . In psychological terms, poor performers lack the metacognitive skills to know they are underperforming; overestimating one's performance appears to be part of underperforming. Our findings may be a result of this effect: the highest antibiotic prescribers (the ‘poor performers’) were more likely to believe they were following the guidelines and, thus, overestimate their own guideline familiarity.
In the medical literature, we are not the first to note a lack of association or even an inverse relationship between self-reported familiarity or knowledge of guidelines and actual performance. Others have found that awareness of and agreement with a guideline does not guarantee knowledge of a guideline or adherence to the guideline  and that clinicians are challenged to assess their own level of knowledge . Clinicians systematically overestimate their own performance compared with objective assessment by an absolute difference of about 25% . For example, consonant with the psychological literature, clinicians overestimated their adherence to hypertension guidelines, and clinicians with low guideline adherence were much more likely to overestimate their adherence to medication recommendations and blood pressure targets . With respect to ARIs, senior medical students’ compliance on hypothetical case vignettes was not correlated with their reported reading of pediatric principles of judicious antibiotic use . Our findings should be added to the literature that shows self-assessment and peer review of performance are, as compared with measuring actual performance, of questionable value [42, 43]. Our findings are particularly striking in that there was greater antibiotic prescribing for almost every type of ARI—both antibiotic-appropriate and antibiotic-inappropriate diagnoses—among clinicians who reported greater familiarity with the ARI guidelines.
Our study and analysis have limitations that should be considered. First, our response rate was high (65%) for a survey of mostly physicians, but not all eligible clinicians responded to the survey. Non-response could have biased the results either towards or away from the null hypothesis. Second, the survey and visit data were cross sectional. Clinicians may not have responded to the survey and treated ARI patients contemporaneously, but 85% of the survey responses we analyzed were from the first survey. In addition, about 16% of the respondents crossed over on their level of guideline familiarity from the first to the second survey. Interestingly, more respondents crossed from being more familiar to less familiar over just a few months. However, using only earlier survey results, omitting respondents who ‘crossed over,’ and adjusting for date-of-visit did not change the results. Third, we used billing codes, which are not 100% accurate for identifying ARI visits or ruling out concomitant, potentially antibiotic-appropriate diagnoses like chronic obstructive pulmonary disease. We previously found that claims-derived, electronic ARI diagnoses as a group had a sensitivity of 98%, specificity of 96% and positive predictive value of 96% . In addition, this analysis focused on the internal consistency of individual clinicians’ self-reported guideline familiarity, specified diagnoses and their actual antibiotic prescribing practices. Fourth, we are unable to control for other unmeasured confounders or factors that might be associated with self-reported guideline familiarity and antibiotic prescribing for ARIs, like overconfidence. Finally, we measured clinician self-reported familiarity with the guidelines and not actual knowledge of the guidelines. It is possible that clinicians who reported greater familiarity with the guidelines were also more knowledgeable about the guidelines and felt the guidelines did not apply to their patients.
In conclusion, we found that self-reported familiarity with the ACP/CDC guidelines for the antibiotic treatment of ARIs appeared to be associated with increased antibiotic prescribing for patients with ARIs. This should serve as an example in a new domain—outpatient management of acute medical conditions—showing that self-reported familiarity with guidelines should not be assumed to be associated with consistent guideline adherence or higher quality of care. Interventions meant to increase guideline familiarity, especially self-assessed, should not be presumed to improve the quality of care.
This work was supported by the Agency for Healthcare Research and Quality (grant numbers R01HS015169, K08HS014563) and the National Heart Lung and Blood Institute (grant number K08 HL072806). In the past 3 years, Dr Linder has received research grant funding from Roche to study antiviral medication prescribing for influenza and Pfizer to study electronic adverse drug event reporting.