|Home | About | Journals | Submit | Contact Us | Français|
Self-report measures of medication adherence are inexpensive and minimally intrusive. However, the validity of self-reported adherence is compromised by recall errors for missed doses and socially desirable responding.
Examined the convergent validity of two self-report adherence measures administered by computerized interview: (a) recall of missed doses and (b) a single item visual analogue rating scale (VAS). Adherence was also monitored using unannounced phone-based pill counts which served as an objective benchmark.
The VAS obtained adherence estimates that paralleled unannounced pill counts. In contrast, self-reported recall of missed medications consistently over-estimated adherence. Correlations with participant characteristics also suggested that the computer administered VAS was less influenced by response biases than self-reported recall of missed medication doses.
A single item VAS offers an inexpensive and valid method of assessing medication adherence that may be useful in clinical as well as research settings.
Monitoring medication adherence poses multiple challenges in clinical and research settings. Several strategies are available for measuring medication adherence, each with their own relative advantages and disadvantages. Table 1 presents the most common methods used for assessing medication adherence in research and clinical practice. While electronic medication monitoring systems offer considerable measurement precision, they are costly and can interfere with commonly used adherence improvement strategies, particularly pill boxes and pocketed dose containers [1, 2]. Alternatively, unannounced pill counts provide accurate information about medications adherence [3–5]. Once again, however, accurate estimates of adherence come at a cost, and unannounced pill counts require significant staff time and patient burden.
In contrast to objective measures of medication adherence, self-report assessments are inexpensive and carry minimal patient burden. State of the science reviews have concluded that self-reported adherence assessments correlate with objective measures including electronic monitoring systems and self-report methods are associated with health indicators, most critically HIV viral load . The most widely used self-report adherence measure was developed by the AIDS Clinical Trials Group (ACTG) network [7, 8]. The ACTG interview asks patients to recall how many doses of their medications they missed in the past week using day-by-day retrospective recall. Self-reported recall of medications missed (SR-recall) is easily integrated into clinical interviews and is relatively non-intrusive. However, self-report adherence measures rely on memory and tend to over-estimate adherence . Asking individuals to recall an event in which they forgot to take their medications has obvious cognitive limitations and can be influenced by socially desirable responding because people with HIV are told routinely to never miss a dose of their medications.
An alternative and even less burdensome self-report adherence measure that has demonstrated promising results is a single item visual analogue rating scale (VAS). The VAS asks individuals to consider a specified time period, such as the previous month, and estimate along a continuum the percentage of medication doses (0% to 100%) that they had taken as prescribed [10, 11]. The VAS correlates in the moderate to high range with both unannounced pill counts and SR-recall [10, 11]. The VAS also significantly correlates with HIV viral load suppression, the endpoint of successful HIV treatment adherence [10–12]. A simple single item measure of adherence is obviously appealing particularly when considered next to the more expensive and burdensome alternatives to measuring medication adherence. The ease of administration offered by the VAS also makes it an attractive measure for clinical settings . Unfortunately, not all studies have reported positive findings from the VAS, with some failing to demonstrate significant associations between the VAS and objective measures of adherence [14, 15].
Another aspect of self-report adherence is the administration procedures. For example, SR-recall measures are increasingly conducted using computerized interviews. Computer administration offers several advantages over face-to-face interviews, including maintaining standardized procedures and automated data collection. Perhaps most attractive is that computers do not send subtle verbal and nonverbal response cues that can influence socially desirable responding during face-to-face interviews [16, 17]. The VAS has been computer administered in research and clinical settings . However, most validity studies of the VAS have used interpersonal interview administration that may have yielded over-estimates of adherence.
The current study was conducted to examine a computerized administration of VAS adherence assessment in relation to SR-recall adherence and an objective measure of medication adherence, specifically unannounced pill counts. In an attempt to minimize socially desirable responding, we administered the VAS and ACTG SR-recall adherence assessments using an audio-computer assisted interview (ACASI). Computerized interviews demonstrate validity for measuring behaviors that are sensitive to socially desirable responding, such as substance use and sexual behaviors . In addition, computerized administration of self-reported adherence measures allow for medication-by-medication assessments that can be averaged across drugs within a regimen. In one study that used computerized administration of the VAS, adherence rates corresponded to the ACTG SR-recall (r = .58) ; adherence estimates were higher on the VAS in comparison to SR-recall, the opposite pattern of what is observed in interpersonal interview administration. The aims of our study were therefore to examine the validity and accuracy of a computer administered VAS for assessing self-reported medication adherence in relation to SR-recall, unannounced pill counts, and common predictors of medication adherence.
One hundred eighty nine men, 94 women, and 15 transgender persons living with HIV/AIDS were recruited from AIDS service organizations, health care providers, social service agencies, and infectious disease clinics in inner-city areas of Atlanta, GA. Recruitment relied on provider referrals and word-of-mouth. Interested persons phoned our research program to schedule a study intake appointment. The study entry criteria were age 18 and proof of positive HIV status and HIV treatment status using a photo ID and matching ARV prescription bottle. All participants were being prescribed ARV medications. Data were collected November 2005 to October 2008.
The core assessments were administered using ACASI procedures. Participants viewed assessment items on a 15-inch color monitor, heard items read by machine voice using headphones, and responded by clicking a mouse. Research has shown that ACASI procedures yield reliable assessments of socially sensitive behaviors that are susceptible to biased responding [16, 17, 20, 21]. Participants were instructed to use the mouse prior to the assessment and were provided with assistance as needed.
Participants were asked their age, years of education, income, ethnicity, and employment status. We assessed HIV related symptoms using a previously developed and validated measure concerning experience of 14 common symptoms of HIV disease. Participants also indicated whether they had ever been diagnosed with an AIDS-defining condition, and their most recent CD4 cell count and viral load.
The VAS for medication adherence was developed as an adjunct self-report measure of medication adherence . The VAS asks individuals to mark a line at the point along a continuum showing how much of each drug they have taken in the past month. For the computerized administration we adapted the response format by using a 100 point slide bar tool anchored by 0%, 50% and 100%. The specific instructions read as follows “We would be surprised if most people take 100% of their medications. Below 0% means you have taken no [name of drug] this past month, 50% means you have taken half of your [name of drug] this past month and 100% means you have taken every single dose this past month. What percent of your [name of drug] did you take?” Participants indicated the percentage of medications taken by clicking their mouse anywhere on the 100 point slide bar continuum. We also administered the VAS during monthly interpersonal interviews conducted over the telephone using the exact same instructions as the computerized interviews.
Participants were interviewed to identify the HIV treatments they were currently taking and entered their regimen into the computerized interview system . Next, the computerized interviewer asked participants to indicate the number of doses taken for each of their medications, one at a time, for each day in the previous week . This section of the interview was repeated for each HIV medication in the participant’s regimen, asking participants to report the same information using the same procedures for the day before yesterday and the day before that and for the remaining four days of the week. The question algorithm used the following script:
How many times a day do you take [name of drug]? You take [name of drug] [response to above] times a day. How many of the [name of drug] pills are you supposed to take at each time? Think about yesterday. Did you miss taking ANY of your [name of drug] yesterday? How many times did you miss taking your [name of drug] yesterday? Now think about the day before yesterday, which would be 2 days ago. Did you miss taking ANY of you [name of drug] the day before yesterday? How many times did you miss taking your [name of drug] 2 days ago? Think about 3 days ago. Did you miss taking any of your [name of drug] 3 days ago? How many times did you miss taking your [name of drug] 3 days ago.
Now think about the entire past week or past 7 days. Other than the previous 3 days which you have already reported on, did you miss taking any of your [name of drug] in those other 4 days?
How many times did you miss taking your [name of drug] in those other 4 days?
The number of doses missed formed the basis for calculating adherence in the past week which was averaged across medications.
Participants enrolled in this study consented to monthly unannounced telephone-based pill counts. Unannounced pill counts have been demonstrated reliable and valid in assessing HIV treatment adherence when conducted in participants’ homes  and on the telephone . Following an office-based training in the pill counting procedure, participants were called at an unscheduled time by a phone assessor. Participants were provided with a free cell phone that restricted use for project contacts and emergency 911. Repeated pill counts occurred over 21 to 35 day intervals and were conducted for each of the ARV medications participants were taking. Pharmacy information from pill bottles was collected to verify the number of pills dispensed between calls. Adherence was calculated as the ratio of pills counted relative to pills prescribed and dispensed. Two consecutive pill counts were necessary for computing adherence. The adherence values reported here represent the percentage of pills taken as prescribed following the office-based assessment.
The ACASI assessment included common correlates of medication adherence including reasons for missing medications, adherence self efficacy, substance use, and depression.
Participants indicated whether they had missed taking any of their medications in the previous month and if so the reasons they attributed to missing medication doses. Twelve common reasons for missing medications were adapted from previous research . Participants may have missed multiple doses and therefore could indicate multiple reasons for missing medications.
We assessed HIV treatment adherence self-efficacy using items adapted from a scale previously reported . This scale was selected because it represents the type of instrument most commonly used to assess self-efficacy . The seven self-efficacy items assessed confidence in one’s ability to overcome barriers to treatment adherence and maintain close adherence to medications, α = .86.
The AUDIT consists of 10 items designed to identify risks for alcohol abuse and dependence . The first three items of the AUDIT represent quantity and frequency of alcohol use and the remaining seven items concern problems incurred from drinking alcohol. Scores on the AUDIT range from 0 – 40 and the AUIDT has demonstrated acceptable internal consistency. Scores of ≥ 8 indicate high-risk for alcohol use disorders and problem drinking.
The DAST-10 is an abbreviated version of the original scale, which was designed to identify drug-use related problems . DAST-10 scores range from 0-10 and the scale is internally consistent, has demonstrated time stability and acceptable sensitivity and specificity in detecting drug abuse.
To assess current emotional distress we administered the Centers for Epidemiological Studies Depression Scale (CESD, ). Participants were asked how often they had experiences and feelings characteristic of depression in the past 7 days, with responses made on the scale 0 = no days, 1 = 1–2 days, 2 = 3–4 days, 3 = 5–7 days. The 20 item CESD yields scores between 0 and 60, with a score of 16 indicating potential depression.
Participants completed informed consent followed by training in how to conduct the unannounced pill counts as described elsewhere . Within three days of the training participants received their first unannounced pill count followed by unannounced monthly calls. Pill count adherence requires two consecutive data points to calculate adherence determined by pills missing relative to pills prescribed. Following two consecutive pill counts, participants returned to the research office to complete the computerized self-report assessment. The retrospective periods for the SR-recall and VAS therefore overlap with medication monitoring determined by pill counts within the same time periods. This research was approved by our institutional review board and was supported by the National Institutes of Health.
Data analyses focused on descriptive statistics for adherence estimates derived from unannounced pill counts (UPC), SR-recall, and the VAS. We examined concordance between the measures of adherence using Pearson correlation coefficients for continuous percent of pills taken and Kappa coefficients for categorical values of adherence. The associations between adherence and viral load were tested using contingency table chi-square statistics. Significance for all analyses was defined as p < .05.
Ninety percent (N = 270) of participants were African-American, 7% (N = 20) were White, and 3% (N = 8) were Hispanic. The average age was 44.7 (SD = 6.7) years old. One in four participants had not completed high school and 61% were disabled or unemployed.
Table 2 presents the medication adherence estimates obtained from UPC, SR-recall, and the VAS. Results showed that SR-recall yielded higher estimates of adherence than both UPC and the VAS. Paired t-tests showed that the difference between UPC and SR-recall was significant, t (297) = 6.6, p < .01, as was the difference between the VAS and SR-recall, t (297) = 9.2, p < .01. The difference between UPC and the VAS was not significant, t (297) = 1.1, p > .1. More than two-thirds of participants indicated 100% adherence on the SR-recall measure compared to less than one-third for the VAS and one in five using UPC. At every level we observed greater adherence from the SR-recall measure than UPC and the VAS, with very similar patterns emerging from UPC and the VAS. Figures 1 through through33 show the frequency distributions of adherence for the three measures. The tendency for SR-recall to result in greater adherence is reflected in the distribution skew of −3.29, standard error (se) = .14, compared to the distribution skew for pill counts of −2.06, se = .14 and the VAS of −2.06, se = .14.
Pearson correlations among the three measures of adherence indicated moderate levels of association: UPC with VAS, r = .48; SR-recall with VAS, r = .58; and UPC with SR-recall r= .34, all significant p < .01. Similar results were obtained for concordance using categorical cut-offs to define adherence. Table 3 shows the resulting Kappa coefficients for adherence defined by categories based on percentages of medications taken. Agreement was moderate between the VAS and SR-recall and between VAS and UPC; Kappa coefficients ranging between .32 and .49. However, agreement between UPC and SR-recall was poor, with Kappa coefficients between .09 and .25.
The telephone interview administered VAS yielded adherence assessments that correlated with the computer administered VAS (r = .70, p < .001). The mean adherence obtained by telephone VAS was 90.8% (SD = 14.3), which is significantly higher than that obtained by computerized interview, t (297) = 8.0, p < .01. With respect to categorical levels of adherence on the telephone interview VAS, 119 (41%) participants stated that they had perfect adherence; resulting in 78% agreement with the computerized administration, Kappa = .54, p < .01. Using 90% as the cutoff, 198 (68%) achieved adherence; 82% agreement with the computer administered VAS, Kappa = .61, p < .01. Overall, 69 (25%) participants indicated 100% adherence on the phone but not on the computer. Similarly, 45 (16%) participants who indicated at least 90% adherence on the telephone reported poorer adherence on the computer.
Overall, 57% of participants reported knowing that their most recent viral load test result was undetectable, 31% indicated their viral load was detectable, and 12% indicated not knowing their viral load. For each measure and at each level of adherence, the association with viral load was significant, although the overall strength of the association was lowest for SR-recall at all levels of adherence (see Table 4).
Table 5 shows the twelve reasons for missing medications indicated by participants with less than 85% of medications taken on each of the three adherence measures. Overall, when lower adherence was estimated by SR-recall, participants endorsed all of the reasons for missing doses more so than when lower adherence was estimated using UPC and VAS. In addition, reasons for missing doses were similar when participant adherence was estimated by UPC and VAS.
As shown in Table 6, estimates of adherence from the three measures varied in their associations with participant characteristics. Adherence assessed by UPC was significantly associated with participant health care, such that having private insurance was related to greater adherence. UPC adherence was also associated with self-efficacy, viral load, CD4 cell counts, HIV symptoms, alcohol and drug use, and depression. Adherence assessed by SR-recall presented a slightly different pattern of associations; SR-recall adherence correlated with older age, income, private insurance, self-efficacy, viral load and CD4 counts, HIV symptoms, drug use and depression. In contrast, the VAS was associated with most of the participant characteristics including age, income, private insurance, number of medications taken, self-efficacy, viral load and CD4 counts, HIV symptoms, alcohol and drug use, and depression.
Two computer administered self-report measures of medication adherence differed in their evidence of validity. We found that the single item VAS yielded adherence rates and a distribution of adherence estimates that paralleled unannounced pill counts and differed from SR-recall. While SR-recall over-estimated adherence relative to pill counts, the values observed from the single item VAS paralleled those from UPC. Compared to UPC the ACASI administered VAS obtained nearly the same mean, median, and proportions of medications taken. Clinically relevant categories of adherence were also similar for the VAS and UPC. Unlike SR-recall adherence, participants with lower VAS adherence and unannounced pill counts reported similar reasons for missing doses. However, participants who were less than 85% adherent on the SR-recall measure reported more reasons for missing medications when compared to both UPC and VAS. The correlation between the ACASI VAS and SR-recall was the same as that reported in previous research . Finally, VAS adherence and unannounced pill counts obtained a similar pattern of correlations with common adherence-related factors. Overall, we found that the VAS yielded results that paralleled those obtained from the far more expensive and burdensome unannounced pill counts.
Computer administration of self-report adherence measures may reduce socially desirable responding as occurs in measuring other sensitive behaviors. While we observed comparable adherence values for the VAS and UPC, adherence from SR-recall remained higher, suggesting socially desirable responding may differentially affect these two measures even when computer administered. Participants were more likely to report 100% adherence on SR-recall than VAS and participants who did report poorer adherence on SR-recall identified more variety of reasons for missing medications. Finally, SR-recall adherence correlated significantly with participant age and income, while neither UPC nor VAS were associated with participant demographic characteristics. Taken together, our findings support past research showing overestimated adherence and potentially biased responses obtained from SR-recall.
The current findings should be interpreted in light of their methodological limitations. Our self-reported adherence measures may have been influenced by social desirability bias. In addition, the UPC and VAS estimates of adherence were determined using a one month time frame whereas the SR-recall relied on a one week time frame in accordance with their standard use. The impact of using different time frames for the adherence measures is not known and could at least in part account for some of the observed differences. Our study was conducted with a convenience sample recruited in one city in the southeastern United States. The findings may therefore not be generalizable to other populations in other regions. With these limitations in mind, we believe that the current findings have important implications for assessing adherence in clinical and research settings.
We found that the single item VAS adherence scale provided comparable estimates of adherence as the far more expensive and burdensome unannounced pill counts. Indeed, the correlations between our computer administered self-report measures were similar to those reported in past research using ACASI administered adherence measures . Trusting a single item measure of a complex behavior such as medication adherence requires a shift away from the convention that single item measures are inherently unreliable and inaccurate. Although researchers will not likely abandon the precision offered by objective measures of adherence, the VAS should be at least considered as an adjunct measure of adherence as it was initially intended . It is also valuable that the VAS reflects patients awareness of their adherence. In clinical settings, however, where UPC may not be feasible for use in routine care, the VAS offers a simple, brief, and inexpensive option for measuring adherence with potentially minimal burden . Future studies are needed to examine the feasibility of computer administered VAS adherence in clinical settings. The current findings are encouraging and when taken together with past research illustrate the VAS should be considered a valid index of medication adherence.