Addressing the question of SPK is a key component in the development of curricula and assessment tools for surgical trainees. Given that an ideal, single objective measurement tool for the assessment of surgical procedural skills continues to elude the surgical community, we took on this project to provide some insight into this question. Spencer12
reported that 75% of the important events in an operation are related to making decisions, whereas only 25% are related to manual skills. This is not a new concept: Frederick Treves wrote in 1891, “The actual manipulative part of surgery requires no very great skill and many an artisan shows infinitely more adeptness in his daily work [...] It is in the mental processes involved in an operation that not a few fail. There is some lack in […] the capacity for forming a ready judgment, which must follow each movement of the surgeon’s scalpel.”13
Our results confirm our expectation that SPK should increase proportionally with level of training and surgical experience. These particular procedures were chosen because of their inherent frequency in the OR, as even entry-level residents are exposed to them on a regular basis. After all, barring breast biopsies and appendectomies, these are the most common operations performed at our centre. The steps of an open appendectomy were chosen as examples for the required task at hand for the residents before engaging in this exercise and, despite being more common than an open hemicolectomy, could thus not be used as part of the assessment tool.
Scores for specific procedures and overall SPK scores increased with level of training, supporting the construct validity of our assessment tool. In addition, with advancement in surgical training, within-group variability in overall SPK scores decreased, indicating a levelling of cognitive ability across members of the same academic rank with increased surgical education and exposure. Medical students displayed similar standard deviation estimates as residents in PGY-2 simply owing to poor performance as a whole cohort, thus displaying low variability among individual results. Finally, by PGY-5, surgeons displayed perfect SPK scores. The procedural knowledge for these basic surgical procedures plateaus, as one would hope, early in surgical training. On our assessment tool, residents in PGY-3 displayed similar scores to those of the staff surgeons. Whether this association holds true for more advanced procedures and whether this knowledge plateau will be maintained at the PGY-3 level as work-hour restrictions impact surgical training remains to be investigated.
There are potentially 2 practical applications of the SPK tool. The first is for curricula planning. The SPK tool can identify and address issues with particular procedures that are not being well taught in a program (e.g., inguinal hernia procedures performed by junior residents). As such, should surgical education curricula need to be reassessed, our tool may offer insight with respect to the current status of procedural knowledge of trainees of a given surgical centre and provide areas of focus on which efforts can be applied. Second, normative SPK scores could be established with additional data, and then the tool could be applied at an individual level, allowing identification of trainees who may need remedial attention to develop this aspect of surgical knowledge.
We believe that ours is the first study to attempt to assess the procedural component of surgical knowledge separately from basic clinical knowledge and surgical judgment or technical knowledge among surgical trainees.1–6
We hypothesize that procedural knowledge should be similar among surgical trainees of the same rank but acknowledge that substantial variance may exist among surgical residents in how they acquire procedural skills. The purpose of our SPK tool is to measure that procedural knowledge once acquired, regardless of how it was obtained. Inherent heterogeneity in operative exposure may also exist within a program with multiple sites and among teaching staff, which may affect the acquisition of SPK.
The following limitations must be acknowledged. The sample size of this pilot study was small and restricted to 1 institution. In addition, the operations for which the SPK tool was developed are common, and this may have masked procedural knowledge among trainees for less common, more advanced procedures. Finally, owing to time constraints, our investigation was conducted as a cross-sectional study rather than longitudinally. Thus, we were not able to assess individual evolution in SPK; however, this would be interesting to evaluate in future studies.