The thematic analysis of the interview data will be presented under a number of main themes. The first two themes “improved classroom atmosphere” and “improved in-class learning” relates to how students felt the individualised ART system influenced their experience of learning in the classroom. Under the theme “anxiety about performance”, students’ anxiety about using the individualised ART system is discussed. This anxiety related to both comparing their own scores to others in the classroom and to feeling nervous about knowing their own results and revealing their knowledge level to lecturers. Under the theme “enhanced self-management of learning”, the multiple ways in which students felt that the availability of individual formative feedback helped them to manage their learning are presented.
In the quoted excerpts, interviewees are labelled “A-G” respectively, and the interviewer is labelled “I”. The removal of an encouraging prompt from the interviewer has been indicated by the use of “…”, and “...” is used to denote the deletion of a section of text for the purpose of brevity.
Improved classroom atmosphere
All students felt that the use of ART enhanced the classroom experience in some way. Many participants felt that the use of ART added a fun and enjoyable aspect to lectures that otherwise contained a lot of difficult content:
"F: I think it sort of lightened the mood as well sometimes when it was really heavy."
Two participants suggested that the use of ART encouraged the student group to bond and become a closer unit by giving the class a common talking point:
"B: It did, you know, bring the group closer together... everyone’s in the same situation and it gives you all something you can talk about, a common interest."
As well as increased interaction between students, many participants felt that the use of ART encouraged a higher level of interactivity between students and lecturers, thereby allowing the lecturer to better gauge students’ progress on particular topics:
"C: I think it’s good for the lecturers as well…because then they can know whether they are teaching it in a way that people can understand or whether people are just going ”ok” and running off and saying I don't understand rather than having the courage to actually say “I don't understand it”."
Improved in-class learning
A number of positive learning outcomes in relation to the use of ART in the classroom were discussed. All participants felt that the use of ART was a helpful way of checking on their progress with the materials being taught:
"A: It gives you an idea at the end of the session if there’s some questions, if you have actually understood what you’ve been told."
As well as helping students to understand their progress, most participants discussed how the use of this system increased their motivation to focus in the lecture:
"B: I think that erm, having the keepads there, keeps you focused because you know you’re going to get asked on them like you’re constantly listening for key points and things so it just keeps you focused."
A further benefit of the use of ART in the classroom was that students valued the opportunity to practice exam-style questions before the real exam:
"B: I think to get used to the style of questions you would be asked they’re really helpful. Because you can parrot-fashion learn things but when you see them in an exam question that twists it slightly…then you panic."
Some students felt that the use of ART to test their knowledge helped them to consolidate any new information while they were in the lecture:
"G: They are a good way of stopping for a minute when you have learned all this stuff all these new concepts it was good to go over it, have a little breather and then carry on again."
Finally, in terms of positive learning outcomes of the use of ART in the classroom, some students discussed how getting the right answer to a question often promoted their confidence in the subject:
"D: If you got an answer right you thought yes I’ve got it sussed, I’m on the right track and it boosted your confidence."
In addition to confidence being promoted when students’ answered a question correctly, participants also discussed how the use of ART also reassured them when they got a question wrong as they could see they were not the only person struggling with the concepts:
"F: I think probably it was quite evenly spread in terms of knowledge and so it gave you the confidence to carry on because you realise actually you are not the only one that didn’t know that."
Anxiety about performance
Negative aspects of ART were also apparent in the interview data. Some participants reported feeling deflated when they got an answer wrong or when they perceived they were in the minority group of students who did not answer a question correctly:
"F: If the majority were right and I was one of the 10% that got it wrong that was horrible, that’s not so good."
Some participants also expressed initial anxiety about testing their knowledge using ART as they were worried about revealing their own level of pharmacology knowledge both to themselves and to the lecturer. Participant F was more worried about becoming more aware of her own lack of knowledge:
"I: And were the nerves related to using the keepads or was there something?"
"F: It was more highlighting the fact that I am stupid that was what I was more nervous about really, just you know, highlighting how little knowledge I have, or had, hopefully."
"I: Highlighting to yourself or others?"
"F: Both really and probably more concerning for myself on a personal level because you know realising your failings or your lack of knowledge or whatever is never a pleasant thing."
Other participants were very concerned about the lecturer’s perception of their knowledge level and this caused them some anxiety. Participant D discusses her initial fears about the tracking of student scores:
"D: I was terrified! I thought this is real big brother and we are going to be watched! So yeah, it increased my anxiety because I thought ‘yeah they are going to see how thick I actually am!’"
For several students, however, their anxiety about answering questions incorrectly lessened over time, as they focused on what they learned from getting a question incorrect:
"A: At first it was a bit like, ohhh, and you beat yourself up about getting the wrong answer but you know, by sort of week two week three you sort of think you know, right...it’s just a learning thing then, rather than feeling “oh God, I got it wrong” or “oh, I got it right”."
For two participants, the confidentiality associated with the use of individualised ART allayed some of their initial anxieties about the system, as they realised that only themselves and the lecturer would have access to their score:
"G: First of all I was a bit apprehensive because I thought oh does everyone else in the room know your results but then I thought the fact that they didn’t, I thought was great."
Enhanced self-management of learning
Interviewees discussed how the receipt of individualised feedback, in the form of an e-mail of their ART results, enhanced their self-management of their learning. All participants described the benefits of receiving an objective reminder of their ART scores so they could keep track of their own progress:
"F: The email feedback really drew it all together and reminded you ok we did this in the class, we had fun but actually here is the bits that you did well and didn’t do so well and I think that was nicely done."
Students felt that the e-mail feedback helped them to identify their learning needs and helped some students direct their revision:
"D: “I was like well these are the areas I need to revise or go back over.”"
While participants found the feedback useful in terms of targeting their learning needs, they did not necessarily change their revision strategy entirely, as most tended to revise all topics regardless of their feedback scores:
"A: I thought well perhaps I need to do more on that subject but then I went through the whole thing anyway you know."
Some students used the feedback sheets as self-assessment tools in preparation for their exams. For example, participant C asked her husband to test her on the questions using the ART feedback sheet:
"C: I got my husband to read them out and see which ones I got right and the ones I didn't get right I went away and looked at and saw what I got wrong and used it that way."
The individualised feedback sheets helped two students decide that they needed to seek extra lecturer support for certain topics:
"D: If the same question came up and I was still getting them wrong I obviously realised that I was either not reading the question right……or else I just had a mental block and I just needed to ask somebody."
When students were asked whether they thought lecturers should contact them to follow-up on their feedback scores to offer assistance, most students felt that this was unnecessary and preferred the current system where they had control over the implications of their individualised feedback scores. Some students felt that if the lecturer did follow-up on their scores and contact them this would cause them anxiety:
"F: I would probably be irritated by that actually and it probably wouldn’t do my confidence any good whatsoever……yes it think for me I would just go into panic mode."
Most students were content with the level of personal control they had to use the individualised feedback in the way they chose to:
"F: At the end of the day we are adult learners so you know you’ve got a problem you can go out and source to find support, you can’t expect to be spoon-fed everything."