Of the 89 (91%) responses received, 53 were from public colleges/schools and 36 from private colleges/schools. The colleges/schools that responded had been in existence a mean of 70.1 ± 50.3 years, with a median of 78.5 years and a range of 2 to 187 years. The majority of the questionnaires were completed by either an assistant or associate dean (52%) or a dean (34%). The remaining responders included chairs of assessment committees (7%), department chairs (2%), curriculum committee chairs (2%), and regular faculty members (2%).
All of the responding schools employed student evaluation of classroom teaching. At the 83 colleges/schools offering APPEs, students also evaluated the experiential teaching provided by APPE preceptors (Table ). Faculty peer evaluation of classroom teaching took place at 59 (66%) of the responding schools, representing a 16% increase from the 50% reported 10 years earlier. Fifteen (18%) schools employed faculty peer review for evaluation of experiential teaching on APPEs, a 6% increase from the 13% reported a decade earlier (Table ).
Comparison of Teaching Evaluation Methods Used in Colleges and Schools of Pharmacy Between 1996 and 2007
Use of evaluation methods other than evaluation by students has undergone a marked increase over the last decade. Formalized self-appraisal of teaching was used by 44 (49%) of the responding colleges/schools, a 32% increase in use since the prior study. Review of teaching portfolios was used by 28 (32%) of the colleges/schools, a 31% increase. Interviews with samples of students occurred at 26 (29%) of the colleges/schools, a 28% increase. Reviews by teaching experts were conducted at 19 (21%) of the colleges/schools, a 20% increase. Alumni evaluated teaching at 9 (10%) of the schools, a 7% increase (Table ).
The percentages of the 89 colleges/schools that evaluated the following groups of faculty members were: tenure track, 91%; non-tenure track, 93%; basic/pharmaceutical sciences, 98%; social and administrative sciences, 96%; pharmacy practice, 98%; paid part-time faculty members, 81%; volunteer faculty members, 55%; and adjunct faculty members, 63%.
Use of Evaluation Results
Self-improvement/development was the most frequently reported use of evaluation results from each method of teaching evaluation, with the exception of evaluation by alumni (Table ). Other primary uses (uses reported by greater than 50% of respondents) of student evaluations of teaching included promotion, tenure, and merit salary increases. Promotion and tenure decisions were primary uses of the results from peer evaluation of classroom teaching, peer evaluation of experiential teaching on APPEs, formalized self-appraisal of teaching, and review of teaching portfolios. Interviews with samples of students and evaluations by alumni were primarily used for curriculum decisions. Promotion was a primary use of reviews by teaching experts.
Procedures for Evaluation by Students
A detailed description of the procedures used for administering student evaluations of classroom teaching is given in Table . At all colleges and schools, student evaluation of classroom teaching was mandatory for faculty members. Furthermore, these evaluations were administered in all required classroom courses and the vast majority (89%) of elective courses. At 48% of the colleges/schools, students evaluated a faculty member's classroom teaching yearly within each required and elective course the faculty member taught. At the remaining colleges/schools, the frequency with which student evaluations of classroom teaching were conducted varied depending on 1 or more of the following factors: a specified number of evaluations that could be conducted per year, whether the course was required, at the faculty's discretion, hours taught by the faculty member teaching the course, and rank of the faculty member teaching the course.
The majority (55%) of colleges/schools conducted student evaluations of classroom teaching at or near the conclusion of the course. Another 35% conducted them at the conclusion of a faculty member's teaching within a team-taught course. A few (3%) colleges/schools conducted an evaluation at both the midpoint and conclusion of a faculty member's teaching within a team taught course, while 7% left the timing to the discretion of the faculty member. In team taught courses, when the teaching of more than 1 faculty member was to be evaluated, separate forms for each faculty member were used at the vast majority (92.0%) of schools.
At the majority of colleges/schools, students completed the classroom evaluations online via computer (54%) or by hand (39%), but participation was not mandatory at 80% of colleges/schools. Students at 94% of colleges/schools were not required to provide their name or an identifying number on the evaluation/survey instrument.
Table provides a description of the procedures used for distributing the results of student evaluations of classroom teaching. More colleges/schools retyped students' written comments (54%) than distributed them in their original form (47%). At most colleges/schools, classroom teaching evaluation results were distributed to the faculty member evaluated, department chairs, and deans (including associate and assistant deans). Distribution occurred at the conclusion of the academic term (ie, semester or quarter in which the course was taught) in 76% of schools. Cumulative comparison data were not provided by 47% of colleges/schools. The remaining colleges/schools provided some comparison data, most commonly aggregate data for the entire didactic program (24%) and aggregate data for faculty members in the same department (20%).
Procedures for the administration and distribution of results of student evaluations of experiential teaching are given in Tables and . Student evaluation of experiential teaching on APPEs was mandatory for preceptors at all schools. At all schools, these evaluations were administered in all required APPEs and the vast majority (95%) of elective APPEs. At 98% of the schools, students evaluated experiential teaching within each required and elective APPE each time they were offered. At the majority of schools, students completed the evaluations at the conclusion of the APPE (99%) using an online format (72%) and completing the evaluation was mandatory (74%). While the majority (66%) of colleges/schools did not require the students to provide a name or identifier, 34% did require this information. Slightly more schools distributed written comments in their original form (53%) rather than retyping them (48%). Evaluation results were distributed primarily to the director of experiential education, the preceptor who was evaluated, and the department chair. Distribution of results occurred at the conclusion of the APPE (36%), conclusion of the academic term (35%), or conclusion of the academic year (27%). The majority (70%) of schools did not distribute cumulative comparison data.
Additional data collected but not shown in Tables and pertained to distance education. Twenty-four (27%) of the responding schools offered a portion of their didactic program via distance learning. Of these 24 schools, 20 (83%) utilized the same forms and procedures for evaluating teaching that were utilized in the on-site program.
Procedures for Evaluation by Peers
A detailed description of the procedures used for peer evaluation of teaching is given in Table . At the majority (62%) of colleges/schools that used faculty peer evaluation of classroom teaching, evaluation was mandatory and occurred yearly (54%). The colleges/schools utilized 1 (45%), 2 (47%), or 3 (8%) faculty reviewers, each of whom conducted either 1 (79%) or 2 (21%) reviews of the faculty peer being evaluated.
Procedural Characteristics for Peer Evaluation of Teaching in Colleges and Schools of Pharmacy
Peer review of experiential teaching on APPEs was mandatory at a majority (67%) of the colleges/schools that used this form of evaluation. The reviews were conducted yearly at 44% of these colleges/schools. The remaining colleges and schools peer reviewed faculty members every 2 years (33%), 3 years (22%), or 6 years (11%). Either 1 (67%) or 2 reviewers (33%) performed the reviews. Each reviewer made either 1 (80%) or 2 (20%) visits.
Table contains descriptive characteristics of the instruments used for teaching evaluations. For student evaluation of classroom teaching, most (46%) colleges/schools developed their own instruments or used one that was employed university-wide (33%). The instruments contain closed-ended items (92%) and open-ended items (70%) for evaluation of the faculty member and course, which were divided into separate sections (54%), with an area for general comments (75%). An ordinal response rating scale was used by all colleges/schools.
The majority (70%) of colleges/schools had developed their own instruments for student evaluation of experiential teaching on APPEs. Some (15%) had adopted instruments from another college/school or acquired instruments through participation in a consortium. The instruments contain closed-ended items (90%) and open-ended items (70%) for evaluation of both the preceptor and the experience; however, less than half (47%) split them into separate sections. The majority of colleges/schools provided an area for general comments (64%). An ordinal response scale was used by all the schools.
The majority (69%) of colleges/schools that used peer evaluation of classroom teaching had developed their own evaluation instruments. The instruments contained both closed-ended items (59% of respondents) and open-ended items (72% of respondents) for evaluation of the faculty member, and an area for general comments (64%). All utilized an ordinal response scale.
Of colleges/schools that used peer evaluation of experiential teaching on APPEs, the majority (85%) had developed their own instruments. For 71% of colleges/schools, the instruments contained open-ended items and an area for general comments. Closed-ended items were used by 43% and varied, with some focusing on the preceptor (43%) and others on the APPE experience (36%). An ordinal response scale was used by the majority (89%) of colleges/schools.