At the start of the study, 756 patients were eligible to participate, of which 228 (30.2%) could not be reached due to wrong numbers or no response upon repeated calls. Of the 528 successfully contacted, 291 (55.1%) agreed to participate in the pilot study and 237 (44.9%) refused. Of those who agreed to participate, the first 99 (34.0%) enrolled had their information sent to the IVR system (Figure ), and the first IVR call was made on April 17th, 2004. However, the pilot study terminated prematurely after two months because of physician and participant complaints. The follow-up interviews were conducted with the first patients enrolled to evaluate the system.
Study population. 1. 756 of the 993 had been called at the time when the pilot study was ended.
Of the 99 participants in the pilot study, approximately half were female (Table ). Participants tended to be older, with over 70% of participants being 70 years of age or greater. The majority of participants (72.7%) spoke French as their first language. On average, each participant had 16 medications dispensed in the first month of the study (April 2004), approximately 6 of which were for the target chronic disease medications. A follow-up interview was successfully completed with 38 of the initial 99 participants (38.4%). Participants of the follow-up interview were more likely to be female, age 80 years or older, and to speak English as their first language, however the differences in age, gender, and language distributions were not great (Table ). Follow-up interview participants also had a greater number of active prescriptions than non-participants, both when considering all types of drugs and when considering targeted drugs for chronic diseases.
Characteristics of the 99 patients who participated in the IVR pilot study
Participant understanding of the purpose of the IVR system
The majority of survey participants experienced some confusion regarding the purpose of the initial IVR system call. Twenty-one (57.6%) participants responded that they did not understand what the call was about the first time the IVR system contacted them (Table ). Participants made comments such as, "I did not understand anything; it was confusing", "I didn't know what this was about at first" and "I did not really understand what it was about because I was not expecting this. It had been a long time since I had signed up for the program." In contrast to these individuals, those who reported understanding the purpose of the IVR system phone calls said that either their physician had explained the pilot study to them or provided them with adequate information about the pilot study.
Participants' opinion regarding the set-up, timing, and accuracy of reminder calls
IVR system technical performance and design
The majority of survey participants (89.5%) found the clarity of the IVR system's voice to be good or fair, and 34 (87.9%) found the volume of the IVR voice to be acceptable. In the open-ended interview, only a few participants made comments that "the words were not very clear or distinct".
The system ran relatively smoothly with regards to setting up the time and location of reminder calls. Among the 28 (73.7%) who had received a call from the IVR system at the time of the interview, 24 (85.7%) reported that they did not have trouble setting up the time and location of their reminder calls.
However, there were many major technical and performance issues reported by participants. Only 6 (21.0%) participants responded that the calls came at the time of day they had requested. Many patients interviewed mentioned they received calls too late in the evening; participants commented "I got a call at 12:00 a.m!", "I received two reminder calls at 1:30 a.m. and did not appreciate that", and "one time, I received a call at 11:00 p.m. and that made me think about dropping the program. That was not considerate." In addition, there were problems with system design: some participants were not happy with the frequency of calls received. In the open-ended interviews, participants reported that the system called to remind them regarding every pill, which for some was "too much" because they were on many medications.
Another major performance issue was that the voice recognition feature of IVR system had difficulty recognizing participants' voices. This caused much frustration for participants, who expressed that "the machine would not catch the yes or the no and sometimes would give an off response", "the machine made you sound stupid because it would ask you to answer even when you already had. It was confusing and would not listen to you and repeat the same thing", and that "it went very quick, did not give you any chances and would just bark something out at you."
Acceptance of IVR technology
It was evident from the analysis of participants' comments from the open-ended questions that many participants were not receptive to the IVR technology. One reason was a frustration with not being able to talk to a real person: "I was irritated because I could not really speak to anybody and/or ask questions", "sometimes the calls were annoying and talking to a machine was not helpful", and "a machine is a machine. There is nothing human about that. I'd rather talk to a real person." In addition, some expressed discomfort with the technology in general, stating "it was intimidating because a computer was talking", "I don't need anything that's automatic", and that "the system was a failure and I had a terrible experience. The person who designed it had no humanistic understanding."
IVR system utility
Twenty (71.4%) interview participants responded that they had already filled their medication prescription at the time they had received the IVR reminder call (Table ). This issue was also identified by comments participants made during the follow-up interview. Participants stated "I don't need to be reminded, I am not senile yet", "we only have a certain number of pills to take and we remember them", and "it (the IVR system) does not do anything for me. It is useless because I take care of it myself. When there are only a couple of pills left, I go to the pharmacy. It is part of my routine." The general attitude of most participants was that they did not have trouble remembering to take medications or refill medication prescriptions. Only 4 participants (14.3%) responded that they found the system helpful for remembering to refill their prescriptions. However, many did indicate that this intervention could be helpful for those who do have difficulties remembering, stating "it is not useful now, maybe later or for others", and that the IVR system "seemed like a good idea for others but not for me."
Comparing those who found the IVR system helpful with those who did not
In an attempt to identify characteristics of those who would find an IVR system most useful for remembering to take medications or refill prescriptions, those who responded that they found the system helpful for remembering to refill prescriptions were compared to those who responded that they did not find the system helpful Although significant differences between the two groups could not be determined due to the small sample size, there appeared to be a trend that participants who found the system helpful were more likely to be older and to have more active prescriptions. In addition, all four participants who found the system helpful understood that the IVR calls were to remind them to take or refill medication the first time the system called, whereas only 9 of the 24 who did not find the system helpful understood the purpose of the first IVR calls. Both groups encountered technical difficulties and expressed frustration with the IVR system in the open-ended interview, but those who found the system helpful for remembering to refill prescriptions made comments such as "(the IVR system) is a good idea, not because I forget but because it is good to be reminded" and "moderately (helpful), keeps track of pills", and also reported fewer negative comments overall than those who did not find the IVR system helpful.