All 31 of the 121 caregivers in the CHARTA Study who owned their own phones were offered participation. One participant decided to withdraw from the parent study because she was concerned that her child’s HIV status might be disclosed to other household members. Ten other participants either no longer had their own phones or had non-functional phones at the time the sub-study began. One participant had a landline and was not included in the analysis; all others had mobile phones, yielding a sample of 19 participants. The median age of participants was 34.0 (IQR 28.5–39.0) years, 90% were female, 79% had completed more than primary school education, and 63% were literate as defined by being able to read and write a sentence in their native language. Randomization of the 12 literate caregivers resulted in six receiving IVR calls and six receiving SMS; all seven illiterate caregivers received IVR calls.
A total of 48 weekly IVR call cycles were initiated over the four-week period (mean of 3.7 weekly IVR calls per participant). Only three IVR calls were made to four participants due to misunderstandings about participant phone numbers or preferred calling times. Six (12.5%) IVR call cycles were incomplete for the following reasons: a persistently busy signal (n=1); a busy signal and network congestion (n=1); a busy signal and intentional call rejection (n=1); and non-answer (n=3). Of the completed IVR call cycles, only six (12.5%) resulted in a successful response. The most common reason for completed yet unsuccessful calls was a hang up during the PIN stage (46%), followed by a hang up or inappropriate response to a research question (29%). One participant changed phone numbers during the study. The median adherence rate obtained was 82% (IQR 71-92%) based on the six successful IVR calls (three IVR calls referred to doses missed in the past seven days; three referred to doses missed in the past 30 days).
Twenty-four SMS cycles were initiated during the study, with four text messages sent to each participant. A total of 76% of the SMS cycles were not answered, meaning that no response was received to the greeting and prompt for the PIN. Detailed reasons for the lack of a response were not available. The median adherence rate obtained was 100% (IQR 99-100%), based on the four successful SMS (two SMS referred to doses missed in the previous 30 days; two referred to doses missed since the last pharmacy refill). One additional response was “do not know”.
See for the types of responses to the IVR calls and SMS by week. There were no detected power outages in the IVR or SMS systems during the study.
Types of responses to the research questions asked during weekly IVR call and SMS cycles.
The concurrent median adherence rates by caregiver three-day report, 30-day VAS, unannounced pill count/liquid weight, and MEMS are shown in .
Median adherence rates (IQR) by different measures. Data reflect a monthly measurement during approximately the same time period in all 19 subjects, except as noted
The following characteristics were assessed as potential predictors of successful IVR calls and SMS: age, sex, primary school education, and literacy. None were statistically significant, although this study was not powered for such an analysis. Age trended toward significance (p=0.16) with a 0.9 odds of a successful call or text message for every year of increasing age.
Qualitative interviews revealed that mobile phones in this population are typically used for personal arrangements, such as making healthcare appointments, brief conversations with family, and setting up business interactions. Calls and/or text messages are both made and received, and most participants share their phones with others. Three participants reported technical challenges with their phones: two had trouble with keeping the battery charged, and the third’s phone frequently dropped calls for unclear reasons. Two participants also reported not keeping their phones with them at all times (e.g. when working in the fields). One participant needed the assistance of a family member to use her mobile phone, but none reported that someone else responded to the study IVR calls or SMS. Most participants had used the dial pad on their phones to load airtime in a pay-as-you-go system; however, none of these participants had experience with PINs or data collection through IVR or SMS prior to this study.
The first theme that emerged from the qualitative interviews was a poor understanding how to respond to the IVR and SMS prompts. Participants expressed confusion and a fear of making mistakes. Several used an incorrect PIN (e.g. their child’s year of birth or their phone number) or hung up on the IVR call or SMS without any input. One participant rejected the IVR calls. Two IVR participants stated:
At first those questions confused me. They would ask you for your year [PIN]. You would still be trying to respond and the call goes off. Then, they say that we shall try to call you back. Yet, the person who had trained me had repeated for me and told me what to do. It really greatly disturbed me. They were hard.
They asked me how many times your child has missed taking his drugs. I got confused with this question because my child had not missed any drug… I did not know that when the child misses a dose, you press two on the mobile phone and when the child has not missed, you press zero. So there I got confused because I did not know what to reply and because of this, they could call me and then I refuse to answer it as I could know that I can’t answer questions I am asked.
Some participants wanted to erase numbers they had entered at one stage of the call or SMS (e.g. the PIN) before entering a response to the next stage (e.g. the research question). Another IVR participant indicated:
Well, for the first time I had forgotten what to do, and when trying to click yes the call went off. They came and trained me again and I learnt that when the call comes and you respond, you don’t first erase, because when I would try doing so the call would go off.
A second theme related to challenges in training. Several participants reported feeling hesitant to acknowledge a lack of understanding during the training sessions. One SMS participant stated:
The one who taught me thought that I understand this thing so much which was not the case. I don’t blame [the research assistant] because she took me to know but I did not know what to do.
Incomplete understanding was also evident in that several participants reported knowing how to complete the calls and text messages appropriately; however, no successful responses were received from these participants.
Participants suggested the following strategies to improve response rates: repeated trainings over time, training in groups so shy participants can learn from each other, testing knowledge from the trainings, having a dedicated mobile phone, calling from an anticipated number at an anticipated time, writing down the PIN, and allowing more time for responses. An SMS participant said:
At first this call was hard where I used to put a wrong PIN number but since I came to know the PIN number, things changed a lot where I could answer the questions asked. So as time went on, this call became easier. I can tell you that now there is no question I can not answer.
Despite these challenges, all participants said that the technologies were acceptable and wanted to provide proper responses. Some participants also felt that the calls and text messages served as an adherence reminder. An IVR participant reported:
It even increased in my memory that the child is supposed to take her drugs every morning and every night.