Teaching Internal Medicine Residents Quality Improvement Techniques using the ABIM’s Practice Improvement Modules

doi:10.1007/s11606-008-0549-5

J Gen Intern Med. 2008 July; 23(7): 927–930.

Published online 2008 May 1. doi: 10.1007/s11606-008-0549-5

PMCID: PMC2517947

Teaching Internal Medicine Residents Quality Improvement Techniques using the ABIM’s Practice Improvement Modules

Julie Oyler, MD, Lisa Vinci, MD, Vineet Arora, MA MD, and Julie Johnson, PhD

Department of General Internal Medicine, University of Chicago, Chicago, IL USA

Julie Oyler, Phone: +1-773-8341808, Fax: +1-773-8347492, Email: joyler/at/medicine.bsd.uchicago.edu.

Corresponding author.

This article has been cited by other articles in PMC.

Summary

Introduction/aim

Standard curricula to teach Internal Medicine residents about quality assessment and improvement, important components of the Accreditation Council for Graduate Medical Education core competencies practiced-based learning and improvement (PBLI) and systems-based practice (SBP), have not been easily accessible.

Program description

Using the American Board of Internal Medicine’s (ABIM) Clinical Preventative Services Practice Improvement Module (CPS PIM), we have incorporated a longitudinal quality assessment and improvement curriculum (QAIC) into the 2 required 1-month ambulatory rotations during the postgraduate year 2. During the first block, residents complete the PIM chart reviews, patient, and system surveys. The second block includes resident reflection using PIM data and the group performing a small test of change using the Plan–Do–Study–Act (PDSA) cycle in the resident continuity clinic.

Program Evaluation

To date, 3 resident quality improvement (QI) projects have been undertaken as a result of QAIC, each making significant improvements in the residents’ continuity clinic. Resident confidence levels in QI skills (e.g., writing an aim statement [71% to 96%, P

.01] and using a PDSA cycle [9% to 89%, P

.001]) improved significantly.

Discussion

The ABIM CPS PIM can be used by Internal Medicine residency programs to introduce QI concepts into their residents’ outpatient practice through encouraging practice-based learning and improvement and systems-based practice.

Key Words: Internal Medicine residents, quality improvement, practiced-based learning and improvement, systems-based practice, practice improvement module

INTRODUCTION

While the Accreditation Council for Graduate Medical Education (ACGME) core competencies include practiced-based learning and improvement (PBLI) and systems-based practice (SBP), incorporating curricula for teaching these principles in Internal Medicine residency programs may be challenging. Other studies have described curriculum developed to teach PBLI and SBP¹^–⁸; however, few of them use an easily assessible curriculum that can be implemented in Internal Medicine residency programs. Furthermore, few of these curricula have been rigorously evaluated using both educational and clinical outcomes.⁹

Simultaneously, recent reports of suboptimal quality of care in the United States¹⁰^,¹¹ and patient expectations that physicans continually assess their personal quality of care¹² have led to changes in the American Board of Internal Medicine’s (ABIM) maintenance of certification (MOC) process. Beginning in 2006, Internal Medicine physicians fufilling MOC requirements could complete a practice performance self-evaluation, applying key components of quality assessment and improvement in their practice through practice improvement modules (PIMs). A recent study showed that the use of the ABIM’s diabetes PIM by practicing physicians resulted in meaningful behavioral change.¹³ The ABIM has also adapted PIMs for use in Internal Medicine residency programs. While there are no published studies documenting the use or effectiveness of PIMs in resident education, a few are currently underway.¹⁴ The ABIM’s PIM may be an accessible tool for the faculty to use to teach residents about quality improvement (QI) and to prepare them for MOC.

AIM

The aim of this educational innovation is to describe a longitudinal curriculum using the ABIM’s Clinical Preventive Services Practice Improvement Module (CPS PIM), which teaches residents quality assessment and improvement techniques and to assess the curriculum’s effect on resident confidence in QI skills.

PROGRAM DESCRIPTION

We have incorporated a quality assessment and improvement curriculum (QAIC), which centers on the use of the ABIM’s CPS PIM, into 2 required 1-month ambulatory rotations during the postgraduate year 2 (PGY2; see Table 1). The residents participating in this curriculum have their continuity clinics at 2 clinical sites. Residents rotate through the ambulatory blocks in groups of 10–15. Each month-long ambulatory rotation includes 4 90-minute QAIC sessions, in addition to the residents’ other outpatient clinical responsibilities and lectures. During the first PGY2 ambulatory block, the residents were introduced to the ABIM’s CPS PIM, which included resident’s web-based¹⁵ completion of 5 chart reviews, 5 patient surveys, and a system survey. The web-based CPS PIM then returns group level quality data on over 25 U.S. Preventive Screening Task Force–based screening indicators, ranging from cancer screening to immunizations.¹⁶ The next segment of the CPS PIM directs residents to reflect on this data and brainstorm ideas for improvement.

Table 1

Structure of Quality Assessment and Improvement Curriculum

In the second PGY2 QAIC block, the residents reviewed the group-level PIM data again and chose 1 low-performing area as the focus for a group QI project. Faculty supervisors helped residents to define QI goals by encouraging projects that were: (1) feasible, (2) manageable for clinic staff, (3) under resident control, and (4) could be completed in the curriculum time frame of 1 month. The residents received lectures about developing aim statements, tutorials about process mapping skills, and finally tools for developing small Plan–Do–Study–Act (PDSA) cycles. Residents worked as a team to: (1) formulate an aim, (2) create a process map, (3) select specific measurements, (4) complete stakeholder interviews, and (5) develop and implement a small project to improve quality in their resident continuity clinic. In addition to the PIM data, which served as a “needs assessment,” the residents chose specific measures for each project. The effectiveness of the interventions was assessed through resident-led chart audits before and after the implementation of the QI initiative. The faculty assisted residents in performing a 2-sample test of proportions to compare the preintervention and postintervention data for each group project.

PROGRAM EVALUATION

From July 2006 to June 2007, 34 PGY2 residents participated in the first and second QAIC blocks. During the first block, the residents each performed a prospective sequential sample of 5 patient chart reviews, asked 5 patients to complete the ABIM CPS PIM Patient Survey, and met with clinic leadership to complete the ABIM Systems Survey. A total of 170 of 170 (100%) chart reviews were performed, and 130 of 170 (76%) patient surveys were collected. During a formal feedback session, the residents received group-level data regarding the quality of care they provided and their patient’s opinions of the quality of care the clinic provided. Group level data included 18 patient demographic data points, 9 outcome measures (e.g., body mass index [BMI]

25, low-density lipoprotein cholesterol at goal), 49 processes of care measures (e.g., height recorded, breast cancer screening performed), and 107 systems enhancements (e.g., record contains an up-to-date medication list, practice provided smoking cessation counseling).

Residents noted suboptimal performance on 2 outcome measures: (1) BMI at goal of less than 25 in only 7 of 170 (4%) charts and (2) tobacco cessation counseling documented in 46 of 154 (30%) of relevant charts. The residents realized that while documentation of weight was exceptional (163 of 170 [96%] charts), height was recorded only 25% of the time (42 of 170), which was a major barrier to calculating the BMI for their patients. Results from the patient survey highlighted that 20% (38 of 117) of patients reported difficulty obtaining medication refills. Using this data, residents were able to brainstorm QI goals including: (1) increasing percentage of charts that had height recorded as a step to increase BMI screening, (2) increasing percentage of patients receiving smoking cessation counseling, and (3) improving the refill process. The CPS PIM then asked residents to develop a plan for improvement and to report results of this improvement.

During the second QAIC block, the residents developed a group project that was implemented in their continuity clinics addressing 1 of the 3 areas of improvement highlighted by the CPS PIM. The 3 projects included: (1) BMI collaborative (January block), (2) tobacco cessation collaborative (March block), and (3) the medication refill collaborative (May block; see Table 2).

Table 2

Title: Summary of Quality Improvement Projects Resulting from Quality Assessment and Improvement Curriculum

To address the poor rate of height documentation and, by default, BMI documentation, the BMI collaborative worked with the nursing staff to integrate height into the triage process, posted BMI charts throughout the clinic, and educated residents on the importance of calculating and documenting BMI. Residents developed an educational module to be used by the clinic preceptors during the first 30 minutes of the continuity clinic, which was set aside for standardized clinic education. The BMI collaborative was able to increase documentation of height from 22 of 206 (11%) of all charts in the week before intervention to 143 of 163(88%) of all charts in the week after intervention (P

.001). Although not identified as a problem, it is noteworthy that dictation of weight also improved from 185 of 206 (90%) to 158 of 163 (97%; P

.008). Ultimately, rates of dictation of BMI improved from 9 of 206 (4%) to 129 of 163 (79%; P

.001).

The tobacco cessation collaborative worked with clinic nursing staff to document smoking status and to give patients a “readiness to quit” handout if the patient was currently smoking. The collaborative also educated residents on tobacco cessation counseling and documentation. The tobacco cessation collaborative improved documentation of smoking status from 31 of 75 (41%) charts before intervention to 56 of 84 (67%) charts after intervention (P

.001).

The refill collaborative worked with the continuity clinic medication refill center staff and the local pharmacies to streamline the refill process for both patients and residents. The key part of their intervention was to educate residents on the proper use of the clinic medication refill center and the importance of having an accurate and up-to-date medication list. The residents found in their preintervention data that many residents changed a medication dose/stopped a medication/started a new medication in the dictated plan but did not make the corresponding change in the dictated medication list that was used by the medication refill center. These outdated medication lists were deemed “inaccurate” as they had the potential to lead to improper refills for patients and increased paperwork for the residents. By educating their peers, the residents were able to decrease the number of “inaccurate medication lists” from 54 of 216 (25%) preintervention charts to 27 of 296 (9%) postintervention charts (P

.001).

Themes that were common to all 3 projects included: (1) interdisciplinary effort between residents and clinic staff, (2) resident education that focused on correct documentation, (3) a publicity campaign, and (4) a patient education component. The group QI projects also resulted in a spillover into the attending clinics and into future QI projects. For instance, the attending clinic’s patients also had height and smoking status documented on clinic vitals sheets. The resident groups tended to build on previously implemented initiatives to improve awareness of their project. For instance, the tobacco cessation and refill QI projects added stickers/reminders to the laminated BMI charts that were posted in cubicles around the clinic.

To evaluate the effect of this curriculum, we administered a previously described [²] self-assessment tool of QI skills to the residents before and after the first 2 blocks of the curriculum. The percent of residents reporting that they were comfortable writing a clear aim statement improved from 71% (24 of 34 residents) to 96% (27 of 28 residents; P

.01, chi-square test). The percent of residents reporting that they were comfortable using a PDSA cycle improved significantly from 9% (3 of 34 residents) to 89% (25 of 28 residents; P

.001, chi-squared test). Preliminary evaluation of resident’s qualitative responses has shown a positive reaction to the QAIC curriculum. The following comments were excerpted from the free-text commentary on the post-assessment:

“It helped me realize that I can change things when they don’t work well, instead of just criticizing the current system.”

“I learned how to break QI projects into small manageable steps.”

“I gained experience using the PDSA cycle which is essential to my research interests.”

DISCUSSION

Teaching Internal Medicine residents PBLI and SBP can be challenging because of limited availability of standardized curricula, inadequate faculty experience, and the complexity of QI, which is dependent on local culture and context. The ABIM’s PIM provides a standardized, web-based, evidence medicine–based, affordable, and efficient tool to teach residents principles of quality assessment. The QAIC discussed above integrates the ABIM’s PIM with basic QI lectures to engage residents in hands-on learning of QI principles. Group QI projects empower residents to reflect on the quality data summarized by the PIM and to make significant changes in their continuity clinic experience.

Resident confidence in QI skills appeared to improve as a result of this curriculum. Moreover, the residents were also able to make statistically significant changes to their practice in the continuity clinic with their group QI projects. Residents were able to disseminate their findings in scholarly work including: an internal resident research day, a hospital quality fair, a monthly Quality Improvement in Progress seminar series, and a regional Society of General Internal Medicine meeting.

Challenges to implementing this curriculum include finding funding, time, and faculty with QI experience. Finding time within preexisting ambulatory blocks and creating PBLI/SBP electives are 2 options for implementing this curriculum into already busy resident schedules. Funding issues included financial resources (about $25 per resident) to buy the ABIM PIM for each resident and funding for QI project support, including photocopying expenses, buying height tools, and laminating BMI charts. Other challenges include developing teams that include both residents and clinic staff members and choosing appropriate measures to document the effect of each QI intervention. Although institutions may not have faculty experienced in QI, the ABIM PIM is a simple tool that inexperienced faculty can adopt to teach QI.

The ABIM’s CPS PIM can be used by Internal Medicine residency programs to introduce QI concepts into their residents’ outpatient practice through encouraging PBLI and SBP. It can also be incorporated into preexisting ambulatory rotations in Internal Medicine residency programs.

Acknowledgments

This study was supported by internal funding from University of Chicago, Graduate Medical Education Committee, and Department of Medicine. We are grateful to Dr. Jim Woodruff, Dr. Eric Holmboe, Dr. Greg Orgrinc, and Ms. Kim Alvarez for their support in pursuing this research. Prior presentations of this data include a plenary session presentation at the National Society of General Internal Medicine Annual Meeting in Toronto, Canada, in April 2007, an oral presentation at the Midwest Society of General Internal Medicine Annual Meeting in Chicago, IL, in September 2007, and an oral presentation at the IHI conference in Orlando, FL, in December 2007.

Conflict of Interest None disclosed.

References

1. Holmboe ES, et al. Teaching and improving quality of care in a primary care internal medicine Residency Clinic. Acad Med. 2005;80(6):571–7. [PubMed]

2. Ogrinc G, et al. Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med. 2004;19:496–500. [PMC free article] [PubMed]

3. Canal DF, et al. Practice-based learning and improvement: a curriculum in continuous quality improvement for surgery residents. Arch of Surg. 2007;142(5):479–82. [PubMed]

4. Djuricich A, et al. A continuous quality improvement curriculum for residents: addressing core competency, improving systems. Acad Med. 2004;79(S10):S65–7. [PubMed]

5. Ziegelstein RC, et al. The mirror and the village; a new method for teaching practiced-based learning and improvement and systems-based practice. Acad Med. 2004;79:83–8. [PubMed]

6. Krajewski K, et al. A quality assurance elective for radiology residents. Acad Radiol. 2007;14:239–45. [PubMed]

7. Weingart SN, et al. Creating a quality improvement elective for medical house officers. J Gen Intern Med. 2004;19:861–7. [PMC free article] [PubMed]

8. Zenni EA, et al. A walk in the patients’ shoes: a step toward competency development system based practice. Ambul Pediatr. 2006;6(1):54–7. [PubMed]

9. Boonyasi RT, et al. Effectiveness of teaching quality improvement to clinicians. JAMA. 2007;298(9):1023–38. [PubMed]

10. Institute of Medicine. Crossing the Quality Chasm. Washington, DC: National Academy; 2001.

11. Mcglynn EA, et al. The quality of healthcare delivered to adults in the United States. N Engl J Med. 2003;348:2635–45. [PubMed]

12. Brennan TA, et al. The Role of Physician Specialty Board Certification Status in the Quality Movement. JAMA. 2004;292:1038–43. [PubMed]

13. Holmboe ES, et al. Promoting physican’ self assessment and quality improvement: The ABIM diabete practice improvement module. J Contin Educ Health Prof. 2006;26:109–19. [PubMed]

14. American Board of Internal Medicine Macy (CoVE) Project. Available at http://www.abimfoundation.org/quality/cove.shtm Accessed January 1, 2008.

15. American Board of Internal Medicine Practice Improvement Modules. Available at https://www.abim.org/residency/residency.aspx Accessed January 1, 2008.

16. American Board of Internal Medicine Clinical Preventative Services Practice Improvement Modules. Available at http://www.abim.org/online/pim/demo.aspx Accessed January 1, 2008.

Articles from Journal of General Internal Medicine are provided here courtesy of
Society of General Internal Medicine