The University of Utah Emergency Medicine Residency Program accepted its first class to begin in the summer of 2005. The program is a 3-year residency program in Salt Lake City, Utah, and is approved for eight residents per class. The program received ACGME approval in the summer of 2004. Given the approval timeline, applicants to our program’s first entering class were unable to use ERAS during the fall application process and instead, were required to submit a separate application file to our residency office. The second class of emergency medicine applicants applied in the fall of 2005 for the entering class of 2006. As the residency program had already been approved by ACGME the previous year, applicants to this class used ERAS.
Our study was a retrospective comparison of the evaluators’ subjective ratings of applicants during these 2 consecutive years: 2004–2005, ‘year 1,’ prior to our program’s participation in ERAS, and 2005–2006, ‘year 2,’ our program’s first year participating in ERAS. We reviewed the files of all applicants for these two application cycles and recorded applicant characteristics and credentials as represented in their applications. The first year class, while unable to participate in ERAS, was required to submit identical information as that required through ERAS, thus ensuring uniformity in the data provided by both applicant pools. The objective of our study was to compare the characteristics between all of those who applied during these 2 years. As such, we reviewed the files of every applicant, not distinguishing between those who were interviewed for the program or those who matched. This study received approval from our institutional review board.
In reviewing applications prior to inviting potential candidates for interviews, the program director and associate program director utilized a standard form to assign scores on scales from 0–5 or 0–10 (5 or 10 = highest score) for select subjective components of the application. These components included: research, volunteer/work experience, letters of recommendation, personal statement, dean’s letter, and the applicant’s potential contribution to their class characteristics (Fig. ).
Program directors agreed on basic guidelines on how to evaluate the different components. In evaluating applicants’ research accomplishments, the rater’s assigned score was based on a pre-established list of criteria reflecting the type and number of research studies and publications in which the applicant was involved. The program directors assigned increasing points based on the number of publications, whether the applicant had a first author publication, and whether these publications were in an emergency medicine journal. Additional grading was based on the program directors’ subjective assessments of the strength of the applicants’ dean’s letters, letters of recommendation, etc., relative to other applications.
Program directors scored applicants’ potential contribution to class characteristics based on how they felt the applicant would later perform in a new residency program. They paid particular attention to their perception of the applicant’s ability to take a leadership role in both developing the program as well as building the program’s reputation throughout the hospital’s other specialties and departments.
Given several factors unique to the residency program’s setting, we considered the additional influence of geography on numbers of applicants and applicant characteristics. Salt Lake City is the home of the Church of Jesus Christ of Latter-day Saints, or LDS or “Mormon” Church. As such, it carries with it geographic appeal to members of the church. Additionally, Salt Lake City’s relative isolation from other emergency medicine programs and its recreational offerings introduce additional factors that may influence emergency medicine residency applicants.
To characterize the potential influence of geography on numbers of residency applicants, we developed a surrogate measure to evaluate possible geographic ties to the state of Utah. Residency applications do not include religious preference, nor do they contain information that would allow us to accurately and consistently determine an applicant’s preference for the region’s recreational offerings. We felt, however, that applicants who had previously lived in the state of Utah may be those most likely to be influenced by these geographic ties. We reviewed all applicants and categorized them as having “geographic ties” to the state if their birthplace or undergraduate college/university was in the state of Utah. We used these two markers of geography as these are the two areas of the application for which we could consistently identify prior applicant residence in the state of Utah.
Applications that were not complete (i.e., missing personal statements, letters of recommendations, etc.) were still considered by the program directors and evaluated based on the information submitted. To determine the differences between the study years, chi-square and t-test statistics were used, with p < 0.05 considered statistically significant (SPSS v. 16.0).