In 1999, the Department of Health and Human Services, Health Resources and Services Administration (HRSA), HIV/AIDS Bureau and the Special Projects of National Significance (SPNS) funded 12 sites to evaluate interventions designed to improve ART adherence among people living with HIV/AIDS. The interventions primarily targeted under-served populations with high rates of comorbidities and barriers to ART adherence. Program locations were geographically diverse and representative of the AIDS epidemic in the US, including four in the Northeast, two in the Mid-Atlantic, two in the South, one in the Midwest, two in the West and one in the Pacific Northwest.
The adherence support interventions were situated in a range of clinical settings including community health and academic medical centers. Sites designed their own adherence interventions, reflecting both best practice and other common approaches for promoting ART adherence used in the US and abroad. The singular and combined interventions that the sites employed included, but was not restrictive to, readiness training, modified directly observed therapy, stages of change interventions, professional case management, peer counseling and pharmacist monitoring.
Although the specific adherence interventions and populations varied across the 12 sites, the program evaluation strategy was common to all sites including the evaluation design and data collection strategy, the set of core data elements, instruments and protocols for assessment of adherence and clinical outcomes. Cross-site evaluation of the core data elements was coordinated by the New York Academy of Medicine’s (NYAM) Center for Adherence Support Evaluation. The Institutional Review Boards at both NYAM and each participating site approved the study procedures, including the informed consent protocol.
Participant eligibility and enrollment
Patients were eligible for inclusion in the cross-site evaluation if they met the following criteria: HIV-infected; age 18 years or older; prospective enrollment in the site adherence support program (though not necessarily on ART at baseline) and ability to complete an in-person interview in English or Spanish. Site staff recruited all individuals meeting these common eligibility criteria as well as each site’s program-specific criteria. Post-hoc estimates of client intervention refusal rates by site ranged from 10 – 30% of all individuals asked to participate. Those who agreed to enroll in the cross-site evaluation completed a separate informed consent. Participants were enrolled between July 1, 2000 and March 31, 2002.
Data collection consisted of individual participant interviews and medical chart abstractions at participating sites. Standardized structured questionnaires developed jointly by the participating SPNS sites and CASE were used to conduct individual participant interviews at baseline and at three, six, nine, and 12 months after enrollment. Follow-up interviews were scheduled at three-month intervals starting from the enrollment interview date. Interviewers from each site received training from CASE staff to ensure standardization. While the majority of interviews and chart abstractions were conducted by healthcare professionals, including primary-care physicians, nurse practitioners, registered nurses, pharmacists, nutritionists and case managers, some data were collected by trained peer counselors and program volunteers.
The core cross-site questionnaire included items about sociodemographics (e.g. age, race/ethnicity, gender, educational level, current housing, employment, income and type of health insurance coverage) and self-reported ART adherence. Chart abstractions from participants’ medical records were conducted at baseline and at three-month follow-up intervals at each site to collect HIV RNA levels and CD4 counts, clinical psychiatric diagnoses, ART medications and adherence to medical appointments. Data collected locally were sent to CASE for data management, quality control and entry into the cross-site database. Data were stored into a Microsoft ACCESS 2000 database. All statistical analyses were performed using SAS software release 8.2 and S-PLUS release 6.2.
The CASE Adherence Index
Based on the results of correlation and principal components analyses, the CASE Adherence Index was developed as a composite (sum) of three self-reported measures of adherence:
Self-reported frequency of ‘difficulty taking HIV medications on time (no more than two hours before or two hours after the time your doctor told you to take it)’. Responses were: never, rarely, most of the time or all of the time.
Self-reported ‘average number of days per week at least one dose of HIV medications was missed’. Responses were: everyday, 4–6 days per week, 2–3 days per week, once a week, less than once a week or never. (Reverse coded for analysis).
Self-reported ‘last time missed at least one dose of HIV medications’. Responses were: within the past week, 1–2 weeks ago, 3–4 weeks ago, between one and three months ago, more than three months ago or never.
The correlations among these three questions—difficulty taking medications on time, average number of days per week missed at least one dose and last time missed at least one dose—were moderate, suggesting that they partially measured different dimensions of adherence (). In a principal component analysis of the three variables (A1, A2, A3), the first principal component explained 69% of the total variation in the three variables and was equal to 0.51*A1+/0.62*A2+0.59*A3. Since the first principal component showed approximately equal loadings on the three factors, we considered the sum as the CASE Adherence Index, with A1 contributing a possible range of one to four points, and A2 and A3 each contributing one to six points (see Appendix
). Composite scores ranged from three to 16 with higher scores indicating better adherence.
Correlations among adherence measures.
AACTG three-day self-report
We compared the CASE Adherence Index and its individual components to a three-day self-report originally developed by Chesney, et al. (2000)
for the AACTG. The three-day self-report consists of a day-by-day recall, assessing the number of doses prescribed and the number of doses missed per medication in the prior three days. An adherence rate was calculated for each ART medication by averaging the number of doses taken divided by doses prescribed for that medication in the prior three days. An overall adherence rate for the regimen (SR3) was obtained by averaging the adherence rates for all of the ART medications in the regimen.
The three-day self-report overall adherence rate was analyzed both continuously and as a dichotomous variable using a cut-off of ≥95% or not, based on findings by Paterson, et al. (2000)
and others, of the significant impact on virologic outcome when adherence falls below 95% (Paterson et al., 2000
; Mannheimer, 2005).
HIV RNA levels
HIV RNA levels were measured in two ways: (1) as a dichotomous variable of 1-log decrease in log HIV RNA from baseline to follow-up level and (2) as a dichotomous variable of undetectable HIV RNA at follow-up. Undetectable HIV RNA was defined as HIV RNA<400 because all of the sites did not have access to ultra-sensitive HIV RNA testing. Individual cases from any site with HIV RNA levels <400 copies/ml at baseline were excluded from the one-log decrease measure.
Change in CD4 cell count was measured for each individual participant by taking the difference between the raw quarterly CD4 counts and the raw CD4 baseline measures.
Two steps were taken to assess the CASE Adherence Index’s reliability and validity. In the first step we estimated the CASE Adherence Index’s degree of sensitivity and specificity to changes in the three-day self-report across four cross-sectional time periods. In the second, we compared the CASE Adherence Index and the three-day self-report’s sensitivity to changes in HIV virologic outcomes and CD4 counts across time.
Cross-sectional data from the baseline and three-, six-, nine- and 12-month interviews and quarterly corresponding HIV RNA levels and CD4 laboratory values were used in these analyses. Only participants on ART at the time of the baseline interview were included in these analyses. For each cross-section, only interviews conducted within a 90-day (±45 days) window of the target interview date were included in that quarter’s analysis. Interviews completed more than 45 days after their due date were treated as interviews for the next quarter. Cases with missing quarterly interviews were excluded from that corresponding quarterly analysis. Laboratory data were assessed at baseline or within a quarter if specimens were collected within 45 days prior to or on the day of the baseline interview or within a 30-day window prior to or after the corresponding follow-up interview date. Cases with missing adherence and lab values were excluded from the corresponding quarterly analyses.
Sample available for analyses
Participants on ART at program entry who completed a baseline interview and had corresponding HIV RNA levels and CD4 count laboratory values formed the starting pool of analyzable participants (n = 1,154). A significant decrease in cases available for analyses occurred at follow-up due to a number of factors, including participant attrition before the three-month follow-up interview (n = 252; 22%), not being on ART at the time of the three-month interview (n = 231; 20%), not having an interview in the appropriate time frame (n = 140; 12%) and the lack of corresponding lab values within the specified window period (n = 7; 0.6%). Thus, data from 524 participants were available for the CASE Adherence Index construction; a total of 524, 439, 315, and 305 participants were used to assess the sensitivity and specificity of the CASE Adherence Index to changes in the AACTG three-day self-report at three, six, nine, and 12 months respectively; a total of 405, 328, 248 and 240 cases were used to assess the relationship between the CASE Adherence Index, the three-day self-report and HIV RNA levels at three, six, nine and 12 months, respectively; for the analysis of changes in CD4 counts after baseline measurement 402, 346, 246 and 244 participants were used respectively at three, six, nine and 12 months.