Our goal was to create a process consistent with evaluation models proposed by Pangaro,8
and use the ABIM 1 to 9 scale. We decided to overlay onto the 1 to 9 scale the “learner, manager, teacher” (LMT) constructs and applied George Miller's “pyramid of learning.” The LMT construct translated the criterion-referenced performance rating into functions that were broadly accepted by the residents and faculty.11
promoted a framework for assessing clinical competence with either standardized patients or in work situations, and proposed 4 levels: knows, knows how, shows how, and does (). At SIU, faculty gives a 1 to 4 rating when residents are unable to meet the expectations of faculty. Knowing,
but not knowing how
is a critical distinction as residents transition from learner to manager. The ABIM form equates a performance rating of 1 to 3 as “may endanger patient safety,” and a 4 as “marginal.” Our interpretation of a learner as not yet ready to perform the duties and responsibilities of a resident/manager is internally consistent with this. We use a 5 to 7 rating when the faculty recognizes the resident as an effective manager who meets the faculty's expectations across the competencies by the end of the rotation. We reserve an 8 rating for residents who show an ability to teach and lead in the subcompetencies. Only the Clinical Competency Committee can award a rating of a 9. It requires a score of 8 on all competencies and publications or another meaningful contribution to the goals of the EIP.
shows the scale as it appears on the end-of-the-month global evaluation forms. No numbers are used; a checkbox indicates only criteria mastered and those not yet met. Faculty writes 2 criteria-based statements for each benchmark or subcompetency. The first criterion describes the expectation for residents at the beginning of the rotation, and the second describes the expectation for residents as they leave the rotation. Faculty evaluates residents only at the end of the rotation, placing the resident along a developmental continuum and documenting his/her progress from learner to manager to teacher/leader. Because a rating of 9, “scholar,” is awarded only by the Clinical Competency Committee, this score was removed from evaluation form.
End-of-Month Global Evaluation Scale for Residents, Southern Illinois University School of Medicine
The following paragraphs describe the levels mentioned on .
Level I–Learner. The baseline or minimum knowledge that faculty expect of residents before they start a rotation (1–4).
Level II–Manager. The knowledge that faculty expects residents to demonstrate when they finish the rotation. These are an extension of Level I but contain more advanced performance criteria (5–7).
Level III–Teacher/Leader. Mastery of all parts of Levels I and II, and the ability to disseminate to others an understanding of the multifaceted correct methods to deliver safe and patient-centered medical care (8).
Level IV–Scholar (not shown on the evaluation form). The promotion of the goals of the EIP and/or publishing in peer-reviewed journals (9).
The LMT overlay allowed us to define more clearly how this scale measures competency. Because each step on the scale represents mastery of the materials representing all prior numbers (a Guttman scale), we could use this method of evaluation to foster constructive developmental
assessment sessions. In a Guttman scale, items have a cumulative property and are arranged from least to most extreme. If a resident fails an item, he/she cannot attain the next higher level until mastering that item.12
incorporated a Guttman scale in his 4-stage reporter-interpreter, manager, and educator (RIME) model, recommending that residents be elevated to the next level on the scale only when the subordinate skills have been mastered and demonstrated.
We eliminated the numbers on the evaluation form to focus residents' attention on personal growth in the field of medicine, rather than the achievement of top scores. The numbers have been retained for our comparative database, for reporting to the ABIM. A goal for our evaluation sessions is for faculty and residents to discuss specific, concrete issues related to the anchoring language to improve the accuracy of the evaluation and the quality of the feedback.