Chart review represents a well-established technique for measuring and continuously improving the quality of health care.28
Academic medical centers have provided benchmarking data by postgraduate year,29–31
and evaluation of personal clinical performance against benchmarks has become an accepted intervention in professional development outside of academic settings.32
Results of this study add to the literature by demonstrating that residents who participated in annual reviews of their own patients’ charts exhibited increased adherence to nationally recognized preventive and chronic disease standards. Residents showed significant improvement over postgraduate year in their rates of administering immunizations (tetanus, pneumococcal), screening for diabetes, cholesterol, and cancer (colorectal, cervical), and screening for behavioral risk factors, as well as in their management of diabetes (LDL, microalbumin, foot and eye care). These findings are consistent with the notion that physicians naturally seek to close gaps in their performance once they become aware of them, especially if those gaps are self-discovered.33
Although our focus has always been on residents’ individual practice-based learning and improvement, differences in compliance rates clearly exist within our residency program and in comparisons between our residency and other residency programs. For example, although all of our residents were exposed to the same curriculum and model for faculty supervision, compliance rates were significantly higher at the VA than at the UW. This might be explained by the fact that practice guidelines at the VA were fully integrated into an established electronic medical record system, and residents had primary responsibility for their patient panels, whereas electronic medical records were unavailable at the UW, and residents had to schedule their patients through their attendings, thereby limiting access to patient information and the ability to monitor quality improvements. It is more difficult to explain how specific clinic practices (e.g., reminder systems, protocols for immunizations) might have led to differential improvements in some screening measures (e.g., cervical cancer) and not others (e.g., breast cancer) or to specify reasons for incremental improvements over time given the retrospective nature of our data. Moreover, the fact that we, like Kern (1990),29
found improved outcomes across residency years where others did not,31
or that our compliance rates appear similar to baseline rates in some studies,22
but higher than those reported in other studies15
suggests potential differences in practice sites that may be too numerous to speculate (e.g., patient populations, frequencies of re-visits to clinic, core curriculum).
Residents consistently reported benefiting from the process of reviewing their patients’ charts. Those who completed the survey typically cited gains in understanding about their medical practices, with some, although not a majority of residents, articulating specific progress toward commitments to change. These findings suggest that the residency program may wish to strengthen PBLI skills in continuity clinics,34
while reinforcing residents’ skills at self-reflection through focused mentorship,35
since residents who fail to engage in reflection or critical reflection25
on this exercise may be less capable in general of addressing their gaps and improving their medical practices.
This paper provided support for the feasibility and practicality of our limited cost method of chart review. In contrast to a more expensive model of quality assessment, we estimate that our method could be implemented in a similar-sized program at an annual cost of about $3,000, based on 15 h per year of faculty time (at $100 per hour), supported by 20 h for a program assistant to monitor compliance (at $15 per hour), and 40 h by a data analyst to compile, analyze, and report program data (at $30 per hour), plus the opportunity costs associated with resident time to complete the chart review.
Inherent limitations in the methodological features of our study must also be recognized. We assumed that the value of spending time looking at one’s own work relative to accepted standards of care outweighed the potential for resident bias in chart audit, but in the absence of a control group and double data extraction, we can neither claim that this PBLI activity changed resident behavior nor can we rule out the possibility that improvement over time reflected additional years of training. Similarly, we integrated practice-based learning into the ambulatory care setting assuming that this type of learning would lead to improved patient care practices, but because we did not actually measure improvements in the quality of patient care, we can not confirm this association. Finally, we laid a foundation for lifelong learning by protecting the chart review process from outside evaluation, thereby encouraging residents to take responsibility for their own foibles. However, absence of outcome-based assessments or other evidence to show that our former residents continued to apply their knowledge to self-improvement in their later medical practice hampered our ability to claim success at establishing lifelong self-regulation.
Conclusion Our method of chart review focused on inculcating a spirit of independent reflective thinking and providing residents with an opportunity to improve their practices based on what they learned. Our data collected over 5 years may have been retrospective, but they resulted in the accrual of a large number of individual measurements, and our findings seem promising enough to recommend more rigorous testing. Indeed, if this practical, meaningful, limited cost PBLI method could be shown to lead to change in resident behavior within controlled studies, it seems likely that many residency programs would choose to implement it. It is our hope that our enhanced counseling practices will likewise move residents along a continuum toward increasing critical self-reflection and commitment to individual and system change, further accelerating residents’ progress toward improved patient outcomes.