The finding that time spent by students on their portfolios incrementally increased with each pharmacy year is to be expected because knowledge related to the expected outcomes increases with each pharmacy year. Thus, the reflection on pharmacy courses and where knowledge was acquired would be more in depth and require more time to process. Another factor likely impacting this outcome is the feedback received from advisors over the years. The students gained understanding of what the advisors expected from their reflections with each annual meeting.
Overall, the students perceived the portfolio process as having less than moderate benefit. However, ratings for each item had a large standard deviation, indicating that responses from those who found little value in the process were balanced by responses from those who found considerable value in the process. The highest mean was the students’ perceived benefit from completing the CV/resume as part of the process, although this was not a significant predictor of perceived overall benefit. Some other components that were individually associated with more positive perceptions of benefit were not seen as significant in the final regression model (). It appears to be somewhat important to students that the advisor is willing to meet with them, supports the portfolio process, and provides helpful feedback. Although students rated help from their advisors as being of moderate benefit to the process, there was considerable variability in response to these items, indicating that additional advisor training might be necessary to improve these outcomes.
The students’ lukewarm assessment on the benefit of the portfolio process was somewhat disappointing, considering the effort put forth by students and advisors, but the process definitely improved the role of the advisor. As with most colleges and schools of pharmacy, there is almost no need for academic advising at our college about which courses to take because almost all of the PharmD curriculum is required. Prior to the portfolio assignment, most students rarely ever visited their assigned advisor or saw the Associate Dean for Student Affairs unless they were in academic difficulty. However, based on the opportunity to work together on the portfolios, there is now a fairly robust relationship between advisor and advisee, and some of the results indicate perceived value in this relationship.
The percentage of students suggesting the need to increase the amount of time spent with their advisors increased according to year in school. While third-year students had completed the portfolio process 3 times, first-year students only had completed it once. The results of comment categories in suggest that the students began to increasingly understand the importance of and to desire advisor involvement with each portfolio completed. Based on the results of a large meta analysis, the role of a mentor or advisor was important in the perceived benefit of the portfolio process.5
Several authors from different studies concluded that lack of support from advisors limited the potential benefit of the portfolio.6-9
Another study suggested that the advisor should be involved in a discussion of the student's weaknesses and plan for improvement.6
The results of this study appear to support these findings to some degree, although variability in the quality of advising about the portfolio may have limited positive perceptions.
Portfolios can enhance students’ understanding of the impact of extracurricular activities, help them identify what they need to learn, identify their strengths and weaknesses, and help them understand how to modify their approach to learning.4
The results presented in support this enhancement of student understanding and suggest that students would perceive the portfolio process as being more beneficial if advisors emphasized the understanding of expected outcomes of the professional program, the impact of extracurricular activities on attaining competencies, what should be learned, self-identification of the student's strengths and weaknesses, and modification of the student's approach to learning.
The advisor's role is to help students initiate and expand their self-reflection in these areas. Researchers at the University of Toronto stated that although completing a portfolio does not come naturally to many students, faculty members incorrectly assume that pharmacy students know how to complete portfolios and that there is little need for the advisor to explain the process.3
These findings seem to be supported by the responses of some students in this survey who expressed a need for greater explanation of the process to improve their understanding and to increase the benefit of completing the portfolio. First-year students specifically had a greater desire for more examples of portfolios compared with second- and third-year students. This makes sense because students in their second or third year had completed the portfolio process once or twice already and had personal examples and experience to which to refer.
A study performed by researchers at the Albany College of Pharmacy and Health Sciences evaluated the efficacy of electronic portfolios, which are becoming more common at colleges and schools of pharmacy.10
It would be helpful to know whether students perceive greater benefit from an electronic portfolio compared with a paper-based approach.
Because this study was conducted at a single college, the ability to generalize to the entire pharmacy student population is limited. Assumptions include that students read the questions completely and responded to them truthfully and that they understood the statements. Because the study was conducted 5 months after completion of the last portfolio, some of the associated issues would not be in the students’ most recent memory. Results may have been slightly different depending on when the survey was conducted relative to the portfolio completion date. It would be interesting to determine whether fourth-year students and those who have graduated have a different opinion of the process.
Based on the results of this study, a number of changes will be made to the portfolio process at the University of Arizona. The results will be provided to the faculty of the college so that they can see the potential impact of the advisor on more positive outcomes. Additional development of the advisors is also planned to enhance advisors’ portfolio-related interactions with students. This will include helping advisors guide student understanding of where learning is occurring and how they might enhance learning if they rate themselves as deficient in a particular competency. The Assessment Committee has decided that, in addition to reflecting on how and where they are achieving competencies, students also will be required to rate their level of achievement in each competency on a scale ranging from novice, to intermediate, to proficient. Although student directions for completing the portfolio imply assessment of competency achievement and lack of achievement (ie, students are told to include short- and long-term plans for gaining additional knowledge in the domains), using a rating scale may help students better reflect on their achievement of the competencies and change their approach to learning when they do not feel competent in a given area. Compilation of these scores also will allow the assessment committee to work with the curriculum committee to address any student-identified gaps in learning.