|Home | About | Journals | Submit | Contact Us | Français|
To compare the performance of self-report instruments assessing adherence to antiretroviral therapy (ART) in patients starting ART for the first time and in a predominately Hispanic population.
Of 184 patients in a prospective observational cohort study of newly diagnosed, minority patients of low socioeconomic status, 54 were given MEMS caps for their boosted PI or NNRTI. They completed a 4-week recall visual analogue scale (VAS), the AACTG 4-day recall instrument, and a 1-month recall qualitative single item every 3 months for up to 18 months in English or Spanish. Electronic pharmacy records recorded refill dates.. Spearman’s correlation coefficients were calculated to compare self-report measures with MEMS data and pharmacy data.
Of 46 patients with MEMS data, mean adherence was 84.7% (SD 35.6) by MEMS, 84.5% (SD 15.1) by pharmacy, 95.4% (SD 11.9) by VAS, 95.8% (SD 17.2) by AACTG, and 87.6% (SD 28.2) by qualitative single item. The correlation coefficient (CC) of VAS with MEMS was 0.37 (p<0.01) and with pharmacy was 0.34 (p<0.01). The CC of the AACTG with MEMS was 0.32 (p<0.01) and with pharmacy was 0.28 (p<0.01). The qualitative single item had a CC with MEMS of 0.24 (p<0.01) and with pharmacy of 0.32 (<0.01). Spanish-speaking patients’ VAS adherence had a CC of 0.40 (p<0.01) with MEMS.
The VAS, AACTG, and qualitative single item measures correlated significantly with MEMS and pharmacy data. Our data support self-administration of the VAS, even in Spanish speakers.
In persons with HIV infection, poor adherence to highly active antiretroviral therapy (HAART) has been associated with incomplete viral suppression,1 increased risk of antiretroviral resistance,2 and decreased survival.3 However, assessing medication adherence can be challenging. Pill counts have been used in research settings but are time-consuming. They may also impact the patient-provider relationship if used in the clinical setting. Pharmacy refill records can be helpful, though they may be impractical if patients obtain their medications from multiple pharmacies. It is also important to note that filling a prescription does not necessarily equate to ingestion of the medication.4 Therapeutic drug monitoring is used in some clinical settings, especially in Europe. In a study by Liechty, et al, an abnormally low drug level had a specificity of 88% for detecting adherence of 90% or less.5 However, drug levels are subject to short-term changes in adherence, and factors other than adherence may affect drug levels (e.g., absorption, drug interactions, and timing of sample collection).6 Electronic drug monitoring using Medication Event Monitoring System (MEMS) caps or other systems has been frequently used in research studies to measure adherence. A computer microchip, embedded in a pill bottle cap, pill box, or other container, records the date and time the container is opened. Electronic drug monitoring is often considered as a “gold standard,” because several studies have shown that its measurements correlate closely with viral load.7;8 However, this method is not feasible in clinical settings due to the high cost and complexities of managing such a system.4
Instruments that assess adherence by self-report are low cost and relatively easy to implement, although they modestly reflect actual adherence. One method relies on a visual analogue scale (VAS), asking patients to place a mark along a linear scale to indicate their percent adherence over the previous four weeks.9 The Adult AIDS Clinical Trial Group (AACTG) 4-day self-report measure requires the patient to report doses missed for each prescribed medication over the last 4 days.10 Lu et al. developed a qualitative single-item measure that asks patients to rate their adherence to all of their antiretrovirals over the previous month.11 The VAS, AACTG, and qualitative single-item measure have not been compared in the same study in a newly diagnosed, HAART-naïve patient population that includes Spanish speakers and low-income patients. We were particularly interested in the VAS because it provides an immediate adherence estimate to the clinician in the form of a percent, in contrast to the AACTG and qualitative measure. It also avoids the possibly judgmental terms “poor” and “very poor” employed by the qualitative measure. Here, we report the performance of the VAS, AACTG self-report, and single-item qualitative measure in this patient population, compare them to both MEMS and pharmacy data, and report on additional data examining the validity of the VAS in this population.
We conducted a prospective observational cohort study in Houston, TX, of patients newly diagnosed with HIV infection. Enrollment into the Attitudes and Beliefs and the Steps of HIV Care study (the Steps Study) began January 2006, with the last patient follow-up in March 2009. Patients aged 18 and older were eligible for the study if they had been diagnosed with HIV within the past three months and had not yet completed an outpatient visit with an HIV primary care clinician. Recruitment took place at the Ben Taub and Lyndon Baynes Johnson General Hospitals; the Michael E. Debakey VA Hospital, and the outpatient clinics of the Harris County Hospital District, including the Thomas Street Health Center, an HIV clinic. Patients from City of Houston clinics for sexually transmitted diseases were referred by City Disease Intervention Specialists. Patients were excluded from the study if they were unable to complete the interviewer-administered surveys in English or Spanish. The target enrollment for the STEPS study was 200 patients.
Patients completed an interviewer-administered survey at baseline and every three months for up to 18 months. The survey was generally completed outside of the clinical setting. Patients who started antiretroviral therapy were asked to list their medications and frequency of dosing and complete the VAS, the AACTG self-report, and the qualitative single-item measure. Patients also completed demographics items and items on HIV risk factors, substance and alcohol abuse, and incarceration history. During the 18-month study period, patients also completed the test of functional health literacy in adults (TOFHLA).12 Beginning May 2008, the VAS was a self-administered instrument, while the rest of the survey remained interviewer-administered. Throughout the duration of the study, research coordinators recorded whether patients required no assistance, assistance with percentages only, or total assistance to complete the VAS.
Fifty-four patients in the Steps study were enrolled in an adherence sub-study and given Medication Event Monitoring System (MEMS) caps for electronic adherence monitoring. Patients were eligible for this sub-study if they were using the Thomas Street Health Center pharmacy (to ensure that we could obtain pharmacy refill data), were responsible for taking their own medications, were willing to not use a pill box for the monitored medicine, and had been on HAART for less than six months. Enrollment was capped at 54 based on power calculations, which determined that a sample of N=54 would be able to detect a correlation coefficient of 0.70 (the magnitude of a correlation observed in another study using the VAS) with 80% power.13;14 MEMS caps were placed on the protease inhibitor (PI) or non-nucleoside reverse transcriptase inhibitor (NNRTI) bottle of a patient’s antiretroviral regimen. Data were recovered from the cap every three months for the duration of the patients’ enrollment in the study. Electronic pharmacy and laboratory data from the Harris County Hospital District were collected on the sub-study patients, and medical records were reviewed. The study was approved by the institutional review boards of Baylor College of Medicine and The University of Texas Health Science Center at Houston. All patients provided written informed consent for both the Steps Study and the adherence sub-study.
The primary aim of the sub-study was to compare the validity of the self-report instruments, particularly the visual analogue scale, with adherence calculated by MEMS and pharmacy data. Adherence using MEMS was defined as observed dose events over expected in 28-day intervals. Gaps in MEMS data were excluded from the analysis if they occurred during a period of hospitalization or incarceration, or if the gap lasted >30 days, as it was assumed that patients either were no longer using their cap or had discontinued their medications. Pharmacy adherence for the MEMS medication was calculated using a medication possession ratio during the 28 days prior to each survey.15 For VAS adherence, the location of the ‘X’ that the participant wrote on the scale was converted to a percent. For example, if they placed an ‘X’ midway between the 50% and 60% marks on the scale, their adherence would be 55%.9 Adherence by AACTG for each of the 4 days prior was calculated as 1-(number of doses missed for the day/number of doses prescribed).16 Responses on the qualitative single-item measure were converted to the following numeric values as was done by Lu, et al.: “excellent” = 100%; “very good” = 80%; “good” = 60%; “fair” = 40%; “poor” = 20%; and “very poor” = 0%.11
Criterion validity for the VAS was assessed by comparing the correlation coefficients for VAS versus MEMS and pharmacy data among patients of different ethnicities, genders, ages, HIV risk factors, education levels, incomes, work status, homelessness status, type of insurance, substance abuse, incarceration histories, health literacy and numeracy scores, and language used (English vs. Spanish). For health literacy, TOFHLA scores between 75 and 100 were considered adequate, 60 to 74 marginal, and 75 to 100 inadequate. The TOFHLA numeracy subscale score was considered adequate if 40 and above and inadequate if less than 40.12
Since the adherence data were not normally distributed, Spearman’s correlation coefficients were calculated to compare self-report measures with MEMS data and pharmacy data.17
Fifty-four participants were enrolled in the adherence sub-study, and 46 adherence sub-study participants contributed at least one MEMS/VAS paired reading. Fifty Steps Study participants on ART were not included in the sub-study because they used other clinics, used other pharmacies, or were started on ART after the sub-study slots were filled. No differences were found in gender, age, race/ethnicity, education level, insurance level, or HIV risk factor between participants in the sub-study and these 50 Steps Study participants. The majority of the 46 adherence sub-study participants were male, between 30 and 50 years old, Hispanic, had no high school degree, and had low incomes (Table 1). Twenty-six patients (57%) were on an NNRTI and 20 (43%) were on a boosted PI. The median baseline CD4 count was 75 K/mm3 (IQR 22,197 K/mm3) and the median baseline HIV viral load was 5.44 log10 c/mL (IQR 5.16, 5.78). Detailed characteristics of the participants are shown in Table 1.
Mean MEMS adherence during 28-day intervals with paired VAS data was 84.7% (SD 35.6) with a median of 89.2% (IQR 78.3, 99.1). Mean pharmacy adherence over the length of the study was 84.5% (SD 15.1) with a median of 92.9% (IQR 80.7, 98.2). Mean VAS score over the same time period was 95.4% (SD 11.9) with a median of 98.3% (IQR 92.0, 99.8). Mean AACTG 4-day self-report adherence was 95.8% (SD 17.2), while the median was 100% (IQR 93.8, 100). The qualitative single-item measure had a mean of 87.6% (SD 28.2) and a median of 95.0% (IQR 75.0–100.0).
The Spearman correlation coefficient between pharmacy and MEMS adherence over the 18-month time period was 0.30 (95% CI 0.14, 0.44; p<0.01). The correlation coefficient between VAS and MEMS adherence overall was 0.37 (95%CI 0.22, 0.50; p<0.01), and ranged between 0.18 and 0.53 at each time point (Table 2). The VAS had a correlation coefficient of 0.34 (95% CI 0.21, 0.46; p<0.01) with pharmacy data over the same time period (Table 2). The qualitative single-item measure had a lower, but still statistically significant correlation coefficient with MEMS data (0.24; 95% CI 0.08, 0.38; p<0.01) and with pharmacy data (0.32; 95% CI 0.18, 0.44; p<0.01; Table 3). The correlation coefficient of the AACTG 4-day self-report measure with MEMS data was 0.32 (95% CI 0.16, 0.45; p<0.01); with pharmacy data it was 0.28 (95% CI 0.14, 0.41; p<0.01). The VAS had a correlation coefficient of 0.58 (95% CI 0.50, 0.64; p<0.01) with the qualitative single-item measure and 0.52 (95% CI 0.44, 0.59; p<0.01) with the AACTG measure. The qualitative single-item measure had a correlation coefficient of 0.45 (95% CI 0.36, 0.53; p<0.01) with the AACTG.
The correlation coefficients of VAS to MEMS data in various subpopulations are shown in Table 4. The correlations were statistically significant in most subpopulations examined. VAS adherence in participants who did not need assistance to complete the VAS had a correlation coefficient of 0.28 (95% CI 0.13, 0.42, p<0.01) with the MEMS data; in those participants who needed help with percentages, the correlation was 0.67 (95% CI −0.07, 0.93, p=0.07), while in participants who needed total assistance, the correlation was 0.33 (95% CI −0.49, 0.84, p=0.43). Participants who self-administered the VAS had a correlation coefficient of 0.49 (95% CI 0.25, 0.67, p<0.01) with the MEMS data, while those participants who did not had a correlation coefficient of 0.29 (95% CI 0.11, 0.45, p<0.01). When we restricted the analysis to the participant’s first encounter with the VAS, the correlation coefficient with MEMS for self-administered VAS was 0.49 (95% CI 0.25, 0.67, p=<0.01; 53 measurements), while for the interviewer-administered VAS the correlation was 0.29 (95% CI 0.11, 0.45, p<0.01; 115 measurements). We could not calculate correlations of adherence to viral load because all patients achieved a viral load of <400 copies/mL by 12 months.
In this study of minority patients of low socioeconomic status recently diagnosed with HIV infection, all three self-reported adherence measures significantly correlated with both MEMS data and pharmacy data. The VAS performed as well as the AACTG and single-item qualitative self-report measures in several age groups tested, in patients with inadequate functional health literacy, and in Spanish-speaking patients.
According to the most recent data, Hispanics account for 18% of people living with HIV in the United States.18 Methods to accurately assess medication adherence in this population are needed. Several factors may affect measuring adherence in Hispanic patients. Hopwood, et al showed that Latinos tend to score higher on social desirability scales compared to Whites due to cultural differences.19 This bias might lead Hispanic participants to overestimate their self-reported adherence more often or to a greater degree than White or Black participants. There may also be differences in self-reported adherence between Hispanics with different levels of acculturation, which we did not measure in this study.20 Immigrant Hispanics remaining in HIV care in the U.S. may have higher adherence due to a healthy immigrant effect. To our knowledge, this is the first study to assess and confirm the validity of the VAS in a cohort of predominately Hispanic patients. Clotet et al. assessed general satisfaction of HIV patients in Spain with a visual analogue scale, but did not use this instrument to assess medication adherence. Our findings support the use of the VAS in clinics and studies enrolling diverse populations with HIV infection.
The VAS requires the respondent to think in percentages.11 In our study, the VAS scores of participants who needed no assistance to complete the VAS correlated well with MEMS data. The correlations between VAS and MEMS data for participants who only needed help with percentages and participants needing total assistance were not statistically significant, likely due to there being only eight measurements in each of these groups. The correlations between VAS and MEMS data were moderate at the participants’ first encounter with the VAS (0.29 to 0.49), though significant. Over the duration of the study, the VAS scores of patients who self-administered the VAS strongly correlated with MEMS scores. Further, the correlation for persons who ever had the VAS interviewer-administered was moderate (0.29) and statistically significant. Together, these data suggest that participants who initially self-administered the VAS did not have difficulty using the VAS, and that most persons can successfully use the VAS.
The VAS and MEMS correlations were unexpectedly poor for women, participants aged 30 to 39 years, participants with a high school diploma or G.E.D., and participants with yearly incomes of $25,000 and above. The correlation was also not high in patients with adequate or marginal functional health literacy and high numeric literacy scores. Social desirability may have affected these populations more than the others, but we cannot test this supposition. It is also possible that these results are confounded, but our sample size is not large enough to support multivariate analyses.
This study has several limitations. Only 54 patients were given MEMS caps, and only 46 out of the 54 patients had at least one VAS and MEMS paired observation period. In addition, although most of the participants’ pharmacy data were available for collection, two of the sub-study participants switched to an outside pharmacy, and, we could not retrieve their records. We used a 28-day time frame to compare the adherence measures, but the ACTG only asks about the last 4 days. Adherence over a month is likely more representative of chronic adherence behavior, and we therefore chose 28 days of MEMS data as the referent. Because of the small sample size and success of currently available ART, all of the study participants achieved a viral load <400 copies/mL, so we could not calculate correlation coefficients between the adherence measures and HIV viral load. The primary focus of our analysis was comparing VAS to MEMS data, while the analyses in subpopulations were conducted post-hoc. Multiple comparisons may have created some Type I error.
Although the correlation coefficients between the different self-report measures were generally low or moderate, albeit statistically significant, they are comparable to results from other studies examining self-reported adherence.21–24 Some participants may have been inclined to place an ‘X’ directly over a hash mark on the VAS instead of using the full range of the linear scale, which could affect the correlations with the continuous MEMS and pharmacy data. Many of the VAS, AACTG and qualitative single item scores were clustered in the higher ranges of scores (i.e.>90%), thus causing variability in scores to be low and decreasing the magnitude of correlation coefficients. This phenomenon is likely due to patients overstating their adherence possibly due to social desirability, thus highlighting why there is no self-report measure widely accepted as appropriate for use in routine clinical care.25 Although MEMS caps, pharmacy data, and unannounced pill counts are considered the gold standards for measuring adherence, they are not feasible to use in clinical care at the time a physician is caring for his or her patient. ART adherence is critical to HIV patient survival, and there is a need for a self-report instrument that can be used in routine care.
This study demonstrates that a visual analogue scale can be considered for use as an adherence measure in newly diagnosed, HAART-naïve patients, including Spanish-speaking patients. The VAS measures a 4-week recall period, which others have suggested may be the optimal recall period.11 It can easily be administered to patients, is not time consuming to complete, and our data support the self-administration of the VAS. Like other self-report instruments, the VAS tends to overestimate adherence, but our data show it can be used in clinics and studies of diverse populations living with HIV.
Supported by NIMH grant R34MH074360, AHRQ grant U18HS016093, the Baylor/UTHouston Center for AIDS Research grant P30AI036211, NIH HIV T32AI07456, and the facilities and resources of the Harris County Hospital District and the Michael E. DeBakey VA Medical Center. Dr. Giordano is a researcher at the Michael E. DeBakey VA Medical Center Health Services Research and Development Center of Excellence, Houston, TX. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.