The qualitative analysis identified categories of data relevant to assessment in PBL tutorials. Two categories were identified for students: assessment difficulties and the role of feedback and self-assessment. Two categories were identified for tutors: assessment difficulties and the role of tutors in student assessment.
The discourse of students
The main categories identified in the students' discourse were "assessment: difficulties" and the role of feedback and self-assessment. Their answers show how learning mechanisms and strategies operate in relation to assessment practices. The difficulties concerning assessment practices in tutorials were a recurring theme, strongly linked to the process of grading performance. For example:
"... nine is excellent... and below five no one knows how to grade... no one knows the meanings of two, three or four..."
"... that was an eight for "expression", a five for "study" and a ten for something else.... so, I guess, either cut down on the 10-points scale or include another criterion: sufficient, insufficient, fair...".
Assessment in tutorials requires that students be capable of providing and receiving critical feedback as well as performing self assessment. This often resulted in students feeling insecure and uncomfortable. This not only reveals the subjective nature of assessment, but also the affective factors that lead to specific attitudes, like corporative issues between students. For example:
"... then I won't give him a bad grade... how can you know that he is good? You can't... there's the label..."
"... because you begin to win a kind of cooperation among people, it's hard to give them a poor grade... you do that when it's really critical."
Some answers reveal the interrelationship between assessment, individual traits and behavior present in tutorials. Once again, the issue of grades was influential in determining behavior and attitudes[12
], particularly the concern that good grades are dependent upon good oral presentation skills. For example:
"... because I, for instance... I am a student who may study, but I don't like talking, I hate talking, I don't feel good talking... now you are forced to talk for two years to get grades... Then I end up talking, but I can't express what I really know, you know?"
"... well... it's been tough... because there's the one who talks the most, there's the one who talks the least, there are some halfway... Are you going to give a ten to those who talk the most, five or six to those who talks the least and you'll give a seven or eight to those in the middle?..."
Although assessment in PBL tutorials is aimed at making students think freely about their own learning process, it can be noticed that they still perceived the assessment process as a ranking tool. This created fear and anxiety despite repeated attempts to stress the long term commitment to learning rather than an exclusive focus on grades. As far as critical feedback is concerned, students tended to refer to the attitude of the tutor during tutorial sessions and the lack of criteria when grades were assigned. For example:
"... I think the way students are motivated needs to be evaluated... I've had a tutor who was unable to assess students and who made the whole tutorial session demotivating... and I've already had a tutor who spoke very correctly about each person, and therefore stimulated..."
"... now, the [tutor] I'm with, for instance, has a reputation for giving low grades, and that's what he actually does... The last [tutor] used to give high grades to the same people... I think that's wrong, absolutely wrong... there has to be some standardization... what has to be said is: 'the profile of student X corresponds to grade Y..."
In some answers, it was possible to recognize that the criticism of the assessment process is related to self-criticism. This suggests that students also play a challenging part in the process since they are not certain how to behave. For example:
"...that's tough too..." then you think: "will I give myself a high grade?..." "How well did I do?" "It's hard... self-assessment is hard... how do I assess?"
The discourse of tutors
The discourse regarding assessment includes issues relevant to critical feedback, self-assessment and the role of tutors in the assessment process. As a whole, the data show that, in theory, tutors understand the goals of assessment. However, in practice they often use the traditional model of assessment. Nevertheless, tutors did try to reflect on their own practice and searched for new strategies regarding assessment practices.
One of the major challenges tutors faced was the difficulty of establishing objective criteria for assessing this type of student performance, especially as far as grading is concerned. For example:
"... what I have most difficulty with, and so do the students, is assessment... to know exactly the weight..."
"It would be necessary to specify each item and reach a consensus about what each item represents. ...For example, the ability to recognize, the ability to critically assess the information that is presented or the source of the information..."
"...because I also find it difficult sometimes to discriminate grade by grade, student by student..."
Some answers suggest that the issue of grades is also a source of concern for the tutors. By realizing the subjectivity of the assessment process, tutors also assumed that students were not using objective criteria for the purpose of peer and tutor assessment. For example:
"... students have this overall difficulty... if they have difficulty assessing their peers, they will also have difficulty assessing their tutors... so, if they are giving high grades to their tutors, they might be giving high grades to their peers as well..."
It should be noted that tutors are aware of the need for students to learn how to assess performance in PBL tutorials. For example:
"... many students are not mature enough..." "...there are students who simply shut their eyes and give ten to everybody".
The key role tutors play in assessment clearly emerges from the interview data. The power to grant grades is viewed as one of their important jobs. Even though they attempted to view assessment as a tool to pass and change students, they still demonstrate controlling attitudes that are typical of the traditional culture of assessment. For example:
"... so, it is a valuable tool, but we need to know how to use it... I rarely give a ten... In order for that to happened, I must feel astounded... zero, I don't give that... zero is only for those who were not present... the little experience that I have, that's the way it is...".
"... here you have to use more technical terms, more adequate... So, therefore, what ends up happening? I rarely give a nine or a ten in tutorial assessments in all the items... very seldom... Usually, my grades range, on average, between five and six..."
The tutor's decision-making power in this regard is central:
"...tutors, because of their influence on the group, find it easier to assess... they are in the position of privileged spectators watching the discussion..."