This article has been Background
Formal assessment of clinical competencies is necessary to ensure that all residents are acquiring important skills and, in the United States, will soon become a requirement for residency programme accreditation by the Accreditation Council for Graduate Medical Education (ACGME). The Eye Surgical Skills Assessment Test (ESSAT), a laboratory‐based surgical skills obstacle course, was developed in response to the need for improved tools for the assessment of surgical skills during residency. The ESSAT has previously been shown to have face and content validity, and in this study we sought to determine its inter‐rater reliability and, to some extent, its construct validity.
Methods
Twenty‐seven content experts (residency programme directors and faculty members involved with resident surgical training) watched videos of a junior resident and senior resident completing the three ESSAT stations (skin suturing, muscle recession, and phacoemulsification: wound construction & suturing technique) and completed assessment forms, both task‐specific checklists and a global rating scale of performance.
Results
The ESSAT showed strong inter‐rater reliability for determining whether a resident “passed” a threshold of competency at each station for both the checklists and global rating scale. In addition, for each station, the senior resident was consistently rated above a “passing” threshold using either assessment form, whereas the junior resident was more often rated below (94% vs 30% passing on completed forms).
Conclusion
These results, along with the findings of our face and content validity analysis, support the reliability and validity of the ESSAT, and indicate that it could be a useful tool for improving the assessment of surgical skill during residency. The ESSAT is a tool that all residency programmes could implement as a part of their ophthalmic surgical curriculum and competency assessment, and may be useful to set a threshold of competence that all residents would need to achieve prior to entering the operating room.