PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Atherosclerosis. Author manuscript; available in PMC 2017 December 10.
Published in final edited form as:
PMCID: PMC5723424
NIHMSID: NIHMS923689

Case-based Educational Intervention to Assess Change in Providers’ Knowledge and Attitudes Towards the 2013 American College of Cardiology/American Heart Association Cholesterol Management Guideline

Yashashwi Pokharel, MD, MSCR,a,b,c,1 Lynne Steinberg, PhD,d Winston Chan, MPH,b,e Julia M. Akeroyd, MPH,b,e Peter H. Jones, MD,a,b,c Vijay Nambi, MD, PhD,a,b,c,f Khurram Nasir, MD, MPH,g Laura Petersen, MD, MPH,b,e Christie M. Ballantyne, MD,a,b,c and Salim S. Virani, MD, PhDa,b,c,e,f

Abstract

Objective

Prior studies have shown provider-level knowledge gaps regarding the 2013 American College of Cardiology/American Heart Association guideline on the treatment of cholesterol and concerns about 10-year atherosclerotic cardiovascular disease (ASCVD) risk estimation. The effect of an educational intervention to mitigate knowledge gaps is unknown.

Methods

We developed a questionnaire and administered it to providers before (pre-training) and after (post-training) a case-based educational intervention across 6 sites in Texas. The intervention highlighted the key recommendations of the 2013 guideline and the differences from the prior guideline mainly using clinical-vignettes. Several practice pertinent items were also discussed.

Results

Most participants were providers-in-training (78%) in internal medicine (68%). Compared to pre-training, the post-training metrics were: 43% vs. 82% for providers’ ability to identify 4 statin benefit groups; 47% vs. 97% for their awareness of the ASCVD risk threshold of ≥7.5% to initiate discussion about risks/benefits of statin therapy; 9% vs. 40% for awareness of differences between the Framingham and the ASCVD risk estimator; 26% vs. 78% for awareness of the definition of statin intensity; 35% vs. 62% for using a repeat lipid panel to document treatment response and adherence; and 46% vs. 81% for confidence in using the ASCVD risk estimator, respectively.

Conclusions

A case-based educational intervention was associated with significant improvements in providers’ knowledge and attitudes towards the 2013 cholesterol guideline, which could be related to the engaging nature of our intervention, using practice pertinent information and clinical vignettes. Such interventions could be useful in effective dissemination of the cholesterol guideline.

Keywords: case-based educational intervention, cholesterol, guideline, improving patient care

Introduction

The 2013 American College of Cardiology/American Heart Association (ACC/AHA) guideline on the treatment of blood cholesterol published in November 2013 made some fundamental changes in the approach to treating blood cholesterol.1 Some of the changes compared with the prior guideline2 included the use of a new 10 year atherosclerotic cardiovascular disease (ASCVD) risk estimator, and a shift in focus from a “treat to low-density lipoprotein cholesterol (LDL-C) target” to a “treat to risk” based approach with moderate-high intensity statin therapy. The 2013 ACC/AHA guideline continued to emphasize the recommendations for lipid testing after initiation of statin therapy to monitor response and adherence as in the prior guideline (national cholesterol education program – adult treatment panel III [ATP-III] report). In addition, the 2013 guideline identified 4 specific patient groups who benefit from statin therapy based on randomized clinical trials.

The 2013 guideline was controversial,36 because of these fundamental deviations in its recommendations compared to the prevailing guideline2 and contemporary practice. There was a concern that using the 2013 guideline significantly more individuals will be eligible for statin therapy,3,4 by as much as 12.8 million, mostly in older adults in the primary prevention group.4 Statin therapy in real world practice has been suboptimal,1113 with one of the important reasons related to gaps in providers’ knowledge and attitudes towards the 2013 cholesterol guideline.14,15 A previous study found that providers have suboptimal knowledge on most knowledge items related to the new cholesterol guideline.15 Therefore, the purpose of the current study was to assess whether a case-based educational intervention, using a conceptual framework,16 primarily targeting provider-level gaps, could improve providers’ knowledge and attitudes towards the 2013 ACC/AHA guideline on the treatment of blood cholesterol.

Materials and Methods

Questionnaire development

Domains pertinent to knowledge and attitude towards the 2013 ACC/AHA cholesterol management guideline were captured in the questionnaire using the Cabana’s conceptual model16 as used in prior studies related to cholesterol management.15,17,18 Knowledge gaps assessed include providers’ familiarity with the 2013 guideline; 10 year ASCVD risk estimator and its difference from the Framingham coronary heart disease (CHD) risk estimator as recommended by the prior ATP-III guideline document; intensity of statin therapy; 4 groups that could benefit from statin therapy (patients with clinical ASCVD; patients with diabetes aged 40–75 years and without clinical ASCVD; patients aged 40–75 years without clinical ASCVD or diabetes and with 10-year ASCVD risk ≥7.5%; and patients with possible familial hypercholesterolemia [FH, i.e., with LDL-C ≥190 mg/dL]). Gaps in attitude included assessment of providers’ agreement with the guideline and 10-year ASCVD risk estimator and the providers’ belief that he/she can perform guideline-recommended care. We also assessed whether a provider believed in repeating a lipid panel in a patient with myocardial infarction (MI) after recently starting statin therapy; and whether he/she used LDL-C as a treatment target. Most questions were presented in a multiple choice format, with some questions presented on a likert scale. Details about the questionnaire have been described separately. (Data in Brief reference)

Questionnaire administration and educational intervention

The questionnaire was refined with the help of a psychometrician and pre-tested in 11 providers. The final questionnaire included 23 items and demographic variables. Approximately 1 year after initial publication of the 2013 guideline, we administered a paper-based questionnaire to internal and family medicine, cardiology and endocrinology providers (n= 150) attending educational conferences at 6 Texas sites between 9/2014–4/2015. The questionnaire included a cover sheet explaining the purpose of the study, informing participants that no identifying information would be collected and that participation was voluntary. There was no remuneration for study participation. Ten to fifteen minutes after handing out the survey, providers were instructed to return their completed surveys (pre-training). A provider who initially refused to fill out the survey or did not return the survey was considered a non-responder.

The educational intervention was based on formal didactics and 7 clinical vignettes to describe the key points of the 2013 guideline and outline its differences from the prior guideline. The ACC/AHA have presentation slides for educational purposes on their websites, and most of the slides in our didactic sessions and clinical vignettes were borrowed from these websites. Attention was given to the evidence behind the new guideline document, the evidence for the shift away from treat-to-target approach, the 4 statin benefit groups, the definition for statin intensity as discussed in the 2013 guideline, and recommendations for lipid testing after starting statin therapy to document adherence and response to statin therapy. We also discussed the process of how the new ASCVD risk estimator estimated risk and the key differences between the ASCVD risk estimator and the Framingham CHD risk estimator as recommended by the ATP-III guideline. Specifically, the 2013 ASCVD risk estimator is race-specific (for Caucasians and African Americans) and assesses 10-year risk of cardiovascular disease (including both fatal and non-fatal MI plus fatal and non-fatal ischemic stroke) as opposed to the Framingham CHD risk estimator (which only provides risk for fatal and non-fatal MI). We also discussed several controversies associated with the new guidelines, such as lack of robust evidence related to some of the recommendations when compared with other recommendations of the new guideline; controversies about the new ASCVD risk estimator; concern for overtreatment with statin therapy in the primary prevention cohort; and lack of evidence didn’t necessarily mean no evidence as it pertained to drift way from LDL-C goal based treatment approach. We then used hypothetical patient cases to illustrate several key points pertinent to the objectives of the current study including the 4 statin benefit groups, identification and treatment of a potential FH patient, discussion of risks and benefits of statin therapy and the importance of assessing patient adherence in lipid management. To make the didactics engaging and interactive, we also discussed several other practice pertinent items related to statin therapy, including the importance of having a discussion regarding benefits and potential harms of statin therapy, potential adverse events, managing statin intolerance, using clinical judgment in starting statin therapy in patients otherwise not belonging to one of the 4 statin benefit groups (such as those with strong family history of premature CHD) and cautioning the use of statin therapy in women of child-bearing age. This content was directly derived from the publicly available educational material from the American College of Cardiology’s educational website.

Several members of the study team delivered the educational intervention using the same set of materials (slides/vignettes) to maintain fidelity of the intervention across sites. When delivering the intervention, we were careful not to explicitly support/reject the 2013 cholesterol guideline. The purpose of the intervention was to elucidate the differences between the two guidelines while reminding providers about the controversies of the new guideline. All of the team members delivering the intervention reviewed and provided feedback on the questionnaire during its development, and agreed on the final version. The same questionnaire that was distributed prior to the didactic session was again distributed (post-training) after the educational session. Providers who did not participate in pre-training did not receive the questionnaire for post-training. The total session duration, including the intervention and completion of the 2 sets of questionnaires, was approximately one hour.

Outcomes

We asked several questions pertaining to the study objectives. Knowledge and practice domain related questions included providers’ awareness of the 4 groups of patients potentially benefiting from statin therapy; 10-year ASCVD risk estimator; outcomes calculated by the new ASCVD risk estimator and its differences from the Framingham CHD risk estimator; definition of statin intensity; providers’ ability to identify a possible FH patient (with LDL-C of 210 mg/dL) for a discussion regarding statin therapy initiation; need for a repeat lipid panel testing after a recent MI; and whether LDL-C should be a target of therapy. Other practice and attitude domain related questions included providers’ awareness of the web version of the ASCVD risk estimator; providers’ feeling comfortable, confident and agreeing with the risk estimator; providers’ assessment of over- or underestimation of the true risk using the risk estimator; and providers’ ability to locate the risk estimator.

Statistical analysis

A Chi-square test was used to compare the proportion of pre- and post- intervention responses for each question. All analyses were conducted using SAS 9.4 (SAS Inc, Cary, North Carolina). The p values were adjusted for multiple comparisons.19 A two tailed p-value <0.05 was considered statistically significant.

The protocol was approved by the Institutional Review Board at Baylor College of Medicine and the Michael E. DeBakey Veterans Affairs Research and Development Committee.

Results

The response rate was 98% (147/150) prior to the intervention (pre-training) and 81% (121/150) after the intervention (post-training). Provider demographic information based on pre-intervention data is shown in Table 1. Most of the study participants were providers-in-training in academic settings. The mean age of the providers was 29.6 years and about 45% were men. For providers-in-practice, the mean time since completion of residency or fellowship training was 9 years. For providers-in-training, the mean amount of post-graduate training was 1.8 years. Approximately 68% of the providers indicated internal medicine as their primary practice specialty.

Table 1
Characteristics of the study population

A comparison of responses related to the knowledge and practice domain questions before and after the educational intervention is shown in Table 2. Prior to the intervention 43.5% of the providers were able to identify the 4 major statin benefit groups, which increased to 82.6% after the intervention (p=0.0002). Prior to the intervention, 47.6% of the providers were aware of the threshold of ≥7.5% 10-year ASCVD risk to initiate discussion regarding risks and benefits of statin therapy, which increased to 97.5% after the intervention (p=0.0002). Providers’ knowledge of the 4 outcomes captured by the pooled cohort ASCVD risk estimator increased from 15.6% before the intervention to 67.8% after the intervention (p=0.0002). Providers’ understanding of the differences between the Framingham 10-year CHD risk estimator recommended by ATP-III guideline and the 10-year ASCVD risk estimator recommended by the 2013 ACC/AHA guideline increased from 8.8% before the intervention to 39.7% after the intervention (p=0.0002). Before the intervention, 32.6% of the providers could identify a possible FH patient and initiate a discussion regarding risk and benefit of statin therapy, which increased to 85.1% after the intervention (p=0.0002). Providers’ understanding of the definition of low, moderate and high intensity statin therapy increased from 25.8% before the intervention to 78.5% after the intervention. Before the intervention, 35.4% of the providers indicated that repeat lipid testing should be performed to assess adherence in a patient with recent MI, which increased to 61.9% after the intervention (p=0.0002). Before the intervention 63.9% of the providers indicated that they use LDL-C as a target of therapy, which decreased to 47.9% after the intervention (p=0.0118). Similarly, 56.5% of the providers incorrectly identified positive family history as one of the variables in the 10-year ASCVD risk estimator on pre-training, which decreased to about 15.7% on post-training (p=0.0002).

Table 2
Comparison of responses to questions in the knowledge and practice domain before and after the educational intervention

A comparison of responses to questions related to the ASCVD 10-year risk estimator before and after the educational intervention is shown in Table 3. Before the intervention, 61.2% of the providers indicated that they were aware of the web version of the risk estimator, compared to 90.1% after to the intervention (p=0.0002). Prior to the intervention, 56.5% of the providers felt comfortable using the ASCVD risk estimator, which increased to 83.4% after the intervention (p=0.0002). Similarly, providers’ confidence in the ASCVD risk estimator increased from 45.6% before the intervention to 81.8% after the intervention (p=0.0002). Although there was no significant difference in providers’ belief that the risk estimator overestimated true 10-year ASCVD risk (28.6% before and 32.3% after the intervention, p=0.5466), more providers indicated that the risk was underestimated after the intervention (11.6% before and 26.4% after the intervention, p=0.0026). Although fewer providers indicated their disagreement with the risk estimator after the intervention (6.1% before and 2.5% after the intervention), the difference was not statistically significant (p=0.2645). Similarly, fewer providers indicated that they did not know where to look for the risk estimator after the intervention (7.5% before and 1.6% after the intervention), and the difference was borderline significant (p=0.0504).

Table 3
Attitude of providers towards ASCVD 10-year risk estimator and their use of 10-year ASCVD risk estimator in their practice

Discussion

To our knowledge, this was the first study to examine whether a case-based educational intervention can increase providers’ knowledge towards the controversial 2013 ACC/AHA cholesterol guideline. We found that a case-based educational intervention helped to significantly change providers’ knowledge and attitudes towards the 2013 guideline. The absolute (and relative) changes in the knowledge and attitude domains after the educational intervention ranged from 16–52% (25–348%), and 15–36% (47–129%), respectively. The downstream effects of these noteworthy changes can lead to important clinical impacts.

Based on prior studies, the effectiveness of educational interventions was, at best, modest (usually <10% absolute change).2023 Some qualities of effective educational strategies include participants’ needs assessment at the beginning of the program; encouraging active participation; and short, yet clear repetition of the key message of the intervention.21,2426 We incorporated all of these important qualities in our educational intervention. The objective of our intervention was to highlight the key recommendations of the 2013 guideline and its deviation from the ATP-III guideline, therefore, providers were aware of their needs assessment. We believe that the case-based discussion was key in highlighting important recommendations/changes pertinent to the 2013 guideline. Furthermore, we discussed a great deal of practice pertinent information, some of which were not included in the questionnaire. Providers found information, such as risk of diabetes associated with statin use, adverse reactions of statin therapy, how to identify and approach a potential FH patient, and how to approach conditions not included in the ASCVD risk estimator (e.g., positive family history) very helpful and engaging. In addition, providers were encouraged to engage in and were given pointers about when a risk discussion with a patient is warranted. Lastly, it was emphasized that clinical judgment should be used along with patient involvement, in situations where clear guidance may not be available (e.g. positive family history of premature cardiovascular disease). About half of the time spent in didactics involved case-based discussion. We feel that all of these unique aspects of the intervention could explain the observed success.

It should be remembered that the purpose of the study was to assess if an informative educational intervention could increase understanding of the 2013 cholesterol guideline, including its strengths and shortcomings. This would allow an informed provider to change their practice behaviors after having better insight into the new guideline and also promote them to seek out overall evidence in cholesterol management.

Several limitations of the study are worth mentioning. The study population was limited to providers attending conferences in the state of Texas. Although there could be regional variation in providers’ knowledge and attitudes towards the 2013 guideline, a prior nationwide survey of providers-in-training did not find such variations in understanding of non-high density lipoprotein cholesterol.18 Therefore, we believe that a case-based educational intervention strategy, such as ours, can be successful in a wider dissemination of the 2013 guideline. Since we performed post-intervention assessment immediately after the educational intervention, we were not able to assess the direct impact of the intervention on providers’ practices, its sustainability and the eventual effect on patient outcomes. However, we believe that a successful strategy, such as the one used in the current study, has the potential to improve practice behaviors. We offered live presentations, which may limit the ability to reach a large number of providers. However, we have previously shown that educational gains related to the ATP-III guideline were similar for internet-based learning programs as compared to live activities.27 Therefore, implementation of a cased-based education intervention related to the 2013 cholesterol guideline in continuous medical educational activities, has the potential to improve providers’ knowledge and attitudes towards the guideline. Since even small changes in providers’ practice behaviors can have significant clinical impact,23 the success achieved with our intervention, if achieved in a larger scale, can have significant downstream impacts on patient outcomes.

Although there were dropouts (n=26) between pre- and post-tests, we do not believe this affected our final results. Dropouts were related to providers not returning the questionnaire, providers leaving the conference site for clinical responsibilities and not returning, and rarely providers unwilling to fill the same questionnaire again after the educational intervention. Given our sample size, we were not powered to perform sub-group analyses for providers-in-training versus providers-in-practice. The 2013 guideline states that there is moderate evidence to support the use of statin therapy for 40–75 years old individuals without diabetes with LDL-C 70–189 mg/dL with 10-year ASCVD risk of 5 to <7.5% for primary prevention, but there is strong evidence in those with 10-year ASCVD risk of ≥7.5%. For the current study, we limited the threshold for discussing statin therapy to 10-year ASCVD risk of 7.5% or higher. Participation in the current study was voluntary in nature, so providers could be more motivated to attend an educational conference than an average general provider, and therefore the pre-training response could be even more unfavorable among the general population of providers. Finally, it should be noted that an educational intervention is just a beginning and a multifaceted intervention addressing other factors, such as administrative and technical components (for e.g. provider detailing, provision of decision support algorithms at the point of care) are important adjuncts for a sustainable quality improvement program.

Conclusions

A case-based educational intervention was associated with significant increase in knowledge towards the 2013 ACC/AHA cholesterol management guideline. Such an approach could help in dissemination and implementation of this guideline in clinical practice.

Acknowledgments

The authors wish to thank Bich N. Dang, MD from Baylor College of Medicine for her input on survey content.

Funding: This work was supported by Baylor College of Medicine Academy of Distinguished Educators Fulbright & Jaworski L.L.P. Educational Grant. Dr. Pokharel is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number T32HL110837. Dr. Virani is supported by the American Heart Association Beginning Grant-in-Aid (14BGIA20460366) and the American Diabetes Association Clinical Science and Epidemiology award (1-14-CE-44). This work was also supported by the Houston VA HSR&D Center for Innovations grant (grant HFP 90-020).

Role of the Funder/Sponsor: The funding sources had no role in study design; in the collection, analysis, and interpretation of data; in writing the report; and in the decision to submit the article for publication.

Abbreviations

ACC/AHA
American College of Cardiology/American Heart Association
ASCVD
atherosclerotic cardiovascular disease
ATP-III
adult treatment panel III
CHD
coronary heart disease
FH
familial hypercholesterolemia
LDL-C
low-density lipoprotein cholesterol
MI
myocardial infarction

Footnotes

Ethical approval

The protocol reference numbers approved by institutional review board at Baylor College of Medicine (approved on 8/15/2014) and Michael E. DeBakey Veterans Affairs Research and Development Committee (approved on 9/8/2014) are H-35309 and 14I06.H, respectively.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Disclosure

Ballantyne: Grant/Research Support- All significant. (All paid to institution, not individual): Abbott Diagnostic, Amarin, Amgen, Bristol-Myers Squibb, Eli Lilly, Esperion, Merck, Novartis, Pfizer, Regeneron, Roche Diagnostic, Sanofi-Synthelabo, Takeda Development Centers of America, NIH, AHA, ADA. Consultant- Abbott Diagnostics, Amarin, Amgen, Astra Zeneca, Eli Lilly, Esperion, Genzyme, Matinas BioPharma Inc, Merck, Novartis, Pfizer, Regeneron, Sanofi-Synthelabo.

Jones: Scientific Advisory Board member for Merck, Amgen and Sanofi/Regeneron. Chief Science Officer for the NLA.

Nambi: Regional Advisory Board: Sanofi Regeneron. Co-investigator on patent filed along with Roche diagnostics for use of biomarkers in prediction of heart failure. Grants: Gulf Coast Medical Foundation, Gillson Longenbaugh Foundation, and Department of Veterans Affairs.

Others: None

References

1. Stone NJ, Robinson JG, Lichtenstein AH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(25 Suppl 2):S1–45. [PubMed]
2. Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III) JAMA. 2001;285(19):2486–2497. [PubMed]
3. Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382(9907):1762–1765. [PubMed]
4. Pencina MJ, Navar-Boggan AM, D’Agostino RB, Sr, et al. Application of new cholesterol guidelines to a population-based sample. N Engl J Med. 2014;370(15):1422–1431. [PubMed]
5. Yeboah J, Sillau S, Delaney JC, et al. Implications of the new American College of Cardiology/American Heart Association cholesterol guidelines for primary atherosclerotic cardiovascular disease event prevention in a multi ethnic cohort: Multi-Ethnic Study of Atherosclerosis (MESA) Am Heart J. 2015;169(3):387–395. e383. [PMC free article] [PubMed]
6. Martin SS, Abd TT, Jones SR, Michos ED, Blumenthal RS, Blaha MJ. 2013 ACC/AHA cholesterol treatment guideline: what was done well and what could be done better. J Am Coll Cardiol. 2014;63(24):2674–2678. [PubMed]
7. Pursnani A, Massaro JM, D’Agostino RB, Sr, O’Donnell CJ, Hoffmann U. Guideline-Based Statin Eligibility, Coronary Artery Calcification, and Cardiovascular Events. JAMA. 2015;314(2):134–141. [PMC free article] [PubMed]
8. Andersson C, Enserro D, Larson MG, Xanthakis V, Vasan RS. Implications of the US cholesterol guidelines on eligibility for statin therapy in the community: comparison of observed and predicted risks in the Framingham Heart Study Offspring Cohort. J Am Heart Assoc. 2015;4(4) [PMC free article] [PubMed]
9. Pursnani A, Mayrhofer T, Ferencik M, Hoffmann U. The 2013 ACC/AHA cardiovascular prevention guidelines improve alignment of statin therapy with coronary atherosclerosis as detected by coronary computed tomography angiography. Atherosclerosis. 2014;237(1):314–318. [PubMed]
10. Rhee EJ, Park SE, Oh HG, et al. Statin eligibility and cardiovascular risk burden assessed by coronary artery calcium score: comparing the two guidelines in a large Korean cohort. Atherosclerosis. 2015;240(1):242–249. [PubMed]
11. Maddox TM, Borden WB, Tang F, et al. Implications of the 2013 ACC/AHA cholesterol guidelines for adults in contemporary cardiovascular practice: insights from the NCDR PINNACLE registry. J Am Coll Cardiol. 2014;64(21):2183–2192. [PubMed]
12. Virani SS, Woodard LD, Ramsey DJ, et al. Gender disparities in evidence-based statin therapy in patients with cardiovascular disease. Am J Cardiol. 2015;115(1):21–26. [PubMed]
13. Pokharel YHR, Nambi V, Shah T, Woodard L, Akeroyd J, Winchester D, Ramsey D, Ballantyne C, Petersen L, Virani S. Statin Use and its Facility-Level Variation in a National Sample of Diabetic Patients. Diabetes. 64(Supplement 1):A101–A187. 691-P.
14. Jame S, Wittenberg E, Potter MB, Fleischmann KE. The New Lipid Guidelines: What Do Primary Care Clinicians Think? Am J Med. 2015;128(8):914 e915–914 e910. [PMC free article] [PubMed]
15. Virani SS, Pokharel Y, Steinberg L, et al. Provider Understanding of the 2013 ACC/AHA Cholesterol Guideline. Journal of Clinical Lipidology. DOI: http://dx.doi.org/10.1016/j.jacl.2015.1011.1002.
16. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–1465. [PubMed]
17. Virani SS, Steinberg L, Murray T, et al. Barriers to non-HDL cholesterol goal attainment by providers. Am J Med. 2011;124(9):876–880. e872. [PMC free article] [PubMed]
18. Negi SI, Steinberg L, Polsani VR, et al. Non-high-density lipoprotein cholesterol calculation and goal awareness among physicians-in-training. J Clin Lipidol. 2012;6(1):50–57. [PubMed]
19. Benjamini Y, Hochberg Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society. Series B (Methodological) 1995;57(1):2289–2300.
20. Akbari A, Mayhew A, Al-Alawi MA, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev. 2008;(4):CD005471. [PMC free article] [PubMed]
21. Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;(2):CD003030. [PubMed]
22. O’Brien MA, Rogers S, Jamtvedt G, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;(4):CD000409. [PubMed]
23. Grol R. Improving Patient Care The Implementation of Change in Health Care. 2. Somerset, NJ, USA: John Wiley & Sons; 2013.
24. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21(3):380–385. [PubMed]
25. Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical education. Evid Rep Technol Assess (Full Rep) 2007;(149):1–69. [PubMed]
26. Zwarenstein M, Hux JE, Kelsall D, et al. The Ontario printed educational message (OPEM) trial to narrow the evidence-practice gap with respect to prescribing practices of general and family physicians: a cluster randomized controlled trial, targeting the care of individuals with diabetes and hypertension in Ontario, Canada. Implement Sci. 2007;2:37. [PMC free article] [PubMed]
27. Fordis M, King JE, Ballantyne CM, et al. Comparison of the instructional efficacy of Internet-based CME with live interactive CME workshops: a randomized controlled trial. Jama. 2005;294(9):1043–1051. [PubMed]