PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
HIV Clin Trials. Author manuscript; available in PMC 2010 September 12.
Published in final edited form as:
PMCID: PMC2937191
NIHMSID: NIHMS229569

Monitoring Medication Adherence by Unannounced Pill Counts Conducted by Telephone: Reliability and Criterion-Related Validity

Abstract

Background

Although demonstrated valid for monitoring medication adherence, unannounced pill counts conducted in patients’ homes are costly and logistically challenging. Telephone-based unannounced pill counts offer a promising adaptation that resolves most of the limitations of home-based pill counting.

Purpose

We tested the reliability and validity of a telephone-based unannounced pill count assessment of antiretroviral adherence.

Methods

HIV positive men and women (N = 89) in Atlanta GA completed a telephone-based unannounced pill count and provided contemporaneous blood specimens to obtain viral loads; 68 participants also received an immediate second pill count conducted during an unannounced home visit.

Results

A high degree of concordance was observed between the number of pills counted on the telephone and in the home (Intraclass Correlation, ICC, = .981, p < .001) and percent of pills taken (ICC = .987, p < .001). Adherence obtained by the telephone count and home count reached 92% agreement, Kappa coefficient = .94. Adherence determined by telephone-based pill counts also corresponded with patient viral load, providing evidence for criterion-related validity.

Conclusions

Unannounced telephone-based pill counts offer a feasible objective method for monitoring medication adherence.

Keywords: HIV//AIDS Treatment, Medication Adherence, Pill Counts, Adherence Assessment, Medication Monitoring

Introduction

Antiretroviral therapies effectively suppress HIV replication and improve the health of people living with HIV/AIDS. Although pharmacological profiles vary, most antiretroviral (ARV) regimens demand at least 90% adherence to achieve optimal clinical outcomes and to avoid development of resistant viral strains. (1) Despite the necessity of maintaining high-levels of ARV adherence, accurate measurement and monitoring of adherence remains a significant challenge in clinical research. Electronic medication monitoring devices underestimate adherence when medications are removed from bottles in multiple doses and electronic devices are incompatible with patient assistance devices, particularly pillboxes and medisets. (2,3) The most common methods for assessing medication adherence, including self-report, provider judgments and tracking pharmacy refills are likely to overestimate adherence. (47)

Office-based pill counts are perhaps among the least reliable means of measuring adherence, achieving only 75% sensitivity for detecting non-adherence to ARVs. (8) The primary sources of error in office-based pill counts stem from patients failing to have all of their pills with them at the time of the count. (9) To resolve the limitations posed by office-based pill counts, Bangsberg and his colleagues (10) developed a home-based pill counting procedure. Unannounced home-base pill counts have demonstrated a high-degree of correspondence with other objective measures of medication adherence including significantly corresponding with plasma viral load (10).

Although unannounced home-based pill counts reliably estimate adherence, they are limited by cost and logistical feasibility, especially in sprawling urban and rural settings. However, these limitations are addressed by an adapted home-based pill counting procedure that uses the telephone to conduct unannounced pill counts in the patient’s home. Kalichman et al. (11) developed an unannounced telephone-based pill count and observed high-degrees of concordance with unannounced home visit pill counts. Initial research on unannounced telephone-based pill count was promising but did not assess validity using an external marker of adherence, such as contemporaneous plasma viral load.

The current study was conducted to examine the reliability and criterion validity of unannounced telephone-based pill counts. We tested the reliability of unannounced telephone-based pill counts by conducting unannounced home-based pill counts immediately following the telephone assessment. We also conducted the first test of criterion-related validity for the telephone-based pill count by obtaining contemporaneous blood specimens and examining the association between telephone-based pill count adherence and plasma viral load.

Methods

Participants

Participants in this study were 68 men and 21 women receiving ARV therapies. We recruited people living with HIV/AIDS for a prospective study using convenience sampling procedures, primarily community referrals and word of mouth.

Measures

Demographic and health characteristics

Participants reported their age, ethnicity, education, income, and related demographic information. To assess HIV-related health status, we asked participants to report the year that they tested HIV positive, whether they had ever been diagnosed with an AIDS-defining condition, and their most recent CD4 cell count. In addition, participants completed a 14-item HIV symptom inventory and reported the number of times they had been hospitalized for an HIV-related medical problem.

Unannounced telephone-based pill counts

Using an adaptation of Bangsberg et al.’s (10, 1213) protocol for unannounced home-based pill counts, we conducted unannounced pill counts by telephone. Participants were called by a trained pill counter at an undisclosed time to count their pills. At an intake session that included informed consent, participants were trained to count their medications using the following steps after answering the telephone: (a) bring all medications that are in the home to a comfortable flat surface near the telephone, including closed bottles, pocketed doses, and pill boxes; (b) sort medications into clusters; (c) select a medication and tell the pill counter the prescription numbers, refill dates, number of refills remaining, and dispensed quantities; (d) report to the pill counter lost or gained pills since their previous count and whether the drug was taken that day; (e) count pills using pharmacist tray and cup provided by the study; if using a pillbox, open each compartment to count the pills without removing them from containers; (f) repeat procedure to double count all pills. The data form used to tabulate the pills counted is shown in the Appendix. All of the data needed to calculate adherence is tabulated in the form, including dates that medications may have been stopped and started between pill counts.

Calculating adherence

Adherence was calculated for the 68 participants who had a previous telephone-based pill count one month prior to the validation study. Specifically, the difference between pills counted at the two times was divided by the pills prescribed, taking into account the number of pills dispensed, pills lost, gained, and taken that day. Stopped medications are adjusted for number of days between the previous pill count and the stop date. Medication refill information, specifically the prescription numbers, filled dates, and remaining number of refills are used to verify the accuracy of medications dispensed over the course of the pill counts. However, the current study did not make adjustments in the pill count data using prescription refill tracking information to provide a conservative test of concordance.

Unannounced home-based pill counts

The home visits could occur any time over the year and was not described as being linked to a telephone assessment. Participants did not know the home visit would occur in conjunction with a telephone assessment. The home visit repeated the same pill counting procedure described above, with home visitors asking again that all pills be present. The home visitor queried whether there were additional pills in the home, specifically pills potentially left in rooms, cabinets, and other places in the home. Home visitors also confirmed whether participants kept their pills in bottles, pillboxes, and pocket dose containers. Participants counted their medications to replicate the exact procedure used during the telephone assessment with the home visitors observing and verifying the pills counted. As occurred during the telephone pill counts, participants counted their pills twice. All of the prescription bottle information collected on the telephone assessments was also collected and verified during the home visits. Home visitors did not communicate with telephone pill counters before, during, or after the pill counts. Adherence was calculated using the previous monthly unannounced telephone-based pill count as described above.

Home observations

Home visitors completed a brief observation record of the home visit, including the surface that participants used when counting pills and whether the participant dropped pills during the home-based count. Home visitors also completed 9-point subjective rating scales for how organized the participant was in counting their pills (1 = very disorganized, 9 = very organized) and how difficult the pill count was for the participant (1 = very difficult, 9 = very easy).

Blood specimen collection and laboratory analysis

Participants were asked to come to the project office to provide blood specimens within 24 hrs of the pill count assessments. Blood samples were provided at the project offices using standard phlebotomy. Whole blood specimens in EDTA tube (Becton Dickinson) were centrifuged at 500 g for 10 min within 4 hr of collection. The plasma was recovered and aliquoted into 1 ml samples and stored at −70°C. Plasma viral load was determined by Roche Amplicor HIV-1 Monitor with sensitivity for detecting down to 50 copies/ml.

Correlates of adherence

We assessed three common correlates of non-adherence (health literacy, substance use, and depression) in relation to discrepant telephone-based and home-based pill counts. (14) Health-literacy was assessed using the reading comprehension scale of the Test of Functional Health Literacy in Adults (TOFHLA, 15), consisting of three standard health-related instruction passages with 50 multiple-choice items completed in a12-min. time limit. As an indicator of global alcohol use, we administered the Alcohol Use Disorder Identification Test (AUDIT; 16), a 10-item self-report instrument that includes quantity and frequency of alcohol use and alcohol-related problems. AUDIT scores range from 0 to 40 (alpha = .91). To assess depression we administered an 11-item version of the Beck Depression Inventory (BDI) that reflects cognitive and affective symptoms of depression responded to along four levels of severity over the previous 7-days. (1718; alpha = .90)

Procedures

Participants provided informed consent and agreed to complete monthly unannounced telephone assessments for one year and one unannounced home visit. During the initial intake session participants were instructed in the monthly telephone-based unannounced pill counting protocol. Training occurred in our offices by an intake assessor and a telephone-based pill counter present on the telephone. Training focused on instructing participants in organizing and counting their pills. Participants were asked about their daily routines to determine acceptable times to attempt assessment calls. Participants were given a cell phone that restricted calls except to emergency 911 and the research project office. The cell phones were provided as a backup to the participants’ personal telephones in order to assure that participants would be able to receive the assessment calls. Participants were also administered an assessment battery using audio computer-assisted structured interviews (ACASI). For the home visits, three teams of home visitors conducted home-based pill counts. At the end of the telephone assessment, the pill counter informed the participant that the home visit they had agreed to at the time of informed consent was set-up for that day and that the home visitor would arrive within 5-minutes of ending their call. Participants were compensated $20 for completing the phone based pill count and another $20 for completing the home-based pill count. All of the study procedures were approved by the University of Connecticut Institutional Review Board and the local oversight management at the AIDS Survival Project in Atlanta.

Data analyses

Reliability

These analyses primarily focused on the concordance between the unannounced telephone-based and home-based pill counts. We examined each of the 157 ARV counts obtained from the 89 participants as well as individual participant level analyses. Pill counts conducted over the telephone and in the home were compared for discrepancy by using the absolute difference between pills counted. For participants with a previous pill count (N = 68), we examined concordance for adherence at the medication level (N = 149 medications) and the individual participant level (N = 68 participants). Concordance was assessed using intra-class correlations. We also examined the categorical levels of adherence classification agreement using Kappa coefficients between adherence defined as the proportion of prescribed pills that were taken, using 5% intervals from 75% to 95% of pills taken. (19)

We also conducted sensitivity tests for the intra-class correlations in total pills counted for participants who did not receive a home visit. Sensitivity analyses were conducted under two scenarios. First, we assumed that the six participants who did not receive a home visit demonstrated a level of non-concordance twice that of the typical participant, using a 10 pill discrepancy between their phone count and the assumed home count. In the second scenario, we assumed the worst discrepancies that we observed, an entire missed supply of one drug, representing a 60 pill discrepancy.

Criterion Validity

Participants were grouped on the basis of their adherence using 5% interval from 75% to 95% categories to examine the association of phone-based pill count adherence in relation to having an undetectable viral load using contingency table chi-square tests. A separate 2 × 2 contingency table was constructed for undetectable/detectable viral load and adherent/non-adherent at varying levels of adherence.

Discrepancy analysis

We tested for differences between participants who had perfectly concordant and those who had any discrepant telephone and home-based pill counts on demographic and health characteristics, adherence correlates, and observations recorded during home visits. Independent t-tests were used for comparisons on continuous measures and contingency table chi-square tests were used for categorical variables. All statistical tests used p < .05 to define significance.

Results

The participant flow through the study is shown in Figure 1. Among the 111 persons screened, 12 were excluded for not taking ARVs. An additional ten participants were unable to be reached by telephone, including their project cell phone, during the week of data collection. Therefore a total of 89 participants completed telephone-based pill counts and provided blood specimens within 24 hrs of the adherence assessments. Of the 89 participants who completed telephone assessments, six were unable to accept the home visit; four stated that they had to leave before the home visitor arrived and two stated they were too busy for the home visit. Three of the participants who were not home visited were men and three were women, four were African American and two were white, and the median age was 49. All of these participants had incomes under $10,000 and none-had graduated high school. Thus, there was no trend to suggest that these participants were different from the average participants in the study. An additional 15 participants had completed a home visit for our previous study (11) and were therefore excluded to avoid sampling redundancy. These participants were included in the viral load analyses because they represent new data. On average, participants had received four pill counts prior to the validation study. There was no association between the number of prior pill counts and adherence.

Figure 1
Participant flow through the study.

The characteristics of the sample are presented in Table 1. Participants were taking a wide range of ARV regimens, with 14 participants taking a single 3-drug combination pill and 27 participants taking a 2-drug combination pill. Nearly one in three (31%) always use a pill box or mediset to sort their pills. The mean adherence calculated from unannounced telephone-based pill counts was 84% (SD = 16.4). Eighty percent of participants were at least 70% adherent to their ARV medications, 66% were at least 85% adherent, 60% were 90% adherent, and 42% were 95% or more adherent. A mean of 2.3 (SD =2.2, range 1 to 13) attempts were required to reach participants by telephone. The average distance between home visit locations was 9.3 miles (SD = 8.2, range 1 mile to 40 miles) and an average of 25.9 min (SD = 9.8, range 5 min to 37 min) was required to conduct pill counts.

Table 1
Characteristics of participants receiving telephone based pill counts (N = 89).

Reliability

The concordances between telephone and home-based pill counts for both raw values of pills counted and adherence at the individual medication and participant levels are shown in Table 2. Overall concordance was high, with intra-class correlations exceeding .93 in each case. Discrepancies for both pills counted and adherence were slight in magnitude. For concordance using clinically meaningful levels of adherence, agreement between telephone-based and home-based pill counts was also quite high (see Table 3). For adherence defined as 75% of pills taken concordance between phone and home pill counts was 93% and for adherence defined as 95% of pills taken there was perfect concordance. Levels of adherence between these two extremes always exceeded 90% agreement, with Kappa coefficients of .94 or greater in each case.

Table 2
Descriptive statistics for pill counts and adherence conducted on the telephone and in participants’ homes.
Table 3
Concordance between unannounced telephone-based and homes-based pill count adherence calculated using four criterion levels of adherence, N =68.

Sensitivity analyses performed with the six participants who did not complete the home-based pill count showed that concordance was robust. Assuming that the six participants would have had a discrepancy of 10 pills, the intra-class correlation was reduced from 0.987 to 0.979, 95%CI 0.966–0.986. Assuming the unlikely and extreme case of 60 pill discrepancies, the intra-class correlation remained high, 0.963, 95%CI 0.943–0.977.

Criterion Validity

Participants with undetectable viral loads had higher adherence (M = 88%, SD=13.7) than those with detectable virus (M= 80%, SD = 19.0), t (87) = 1.97, p < .05. Table 4 shows the adherence for participants with undetectable and detectable viral loads at varying definitions of adherence (75% to 95%). Comparisons showed that in all cases participants who had undetectable viral loads were significantly more likely to be adherent to their medications.

Table 4
Undetectable viral load among individuals who were not adherent compared to those who were adherent at varying definitions of adherence, N= 89.

Discrepancy Analysis

Among the 68 participants with both unannounced telephone and home-based pill counts, 41 (60%) were perfectly concordant. The distribution of pill count discrepancies showed that 9 (13%) pill counts were within one pill, 6 (9%) were two pills, and for 8 (12%) participants there was a three to five pill discrepancy between the telephone and home-based pill counts. Four participants (6%) demonstrated highly discrepant telephone and home-based pill counts. For three participants, pill counts were off by more than 30 pills, with sealed bottles found in the home that were not identified on the telephone. In addition, a fourth participant had a 15 pill discrepancy because an additional pillbox was discovered in the home. However, even these large discrepancies in pill counts had a limited impact on calculated adherence because they only involved one medication. For example, one participant with a pill discrepancy was taking 5 ARV medications, with perfect concordance for the other four medications. The overall adherence for this participant was 66% of pills taken before accounting for the 61 pill discrepancy and 52% adherence after. This participant had a viral load of 310,300, consistent with the observed poor adherence. Similarly, the participant with a 15 pill discrepancy was 49% adherent before the discrepancy and 36% after the identified pills were added and this participant had a viral load of 100.

Participants with perfectly concordant pill counts did not differ from those with discrepant pill counts on any demographic or health characteristic as well as health literacy, alcohol use and depression (see Table 4). However, participants who had discordant telephone and home-based pill counts did differ in their carrying doses of medications with them when they go out, their use of pillboxes, whether they stored their medications in an open visible place, and whether they had dropped pills during the home visit. Home visitors rated participants who had perfectly concordant telephone and home-based pill counts (M=8.4, SD=0.8) as having an easier time counting their pills than participants with discordant pill counts (M=7.8, SD=1.3), t(66) = 2.2, p < .05. In addition, participants with concordant pill counts (M=8.5, SD=0.8) were perceived as more organized when counting their pills than participants with discordant pill counts (M=7.6, SD=1.4), t(66) = 2.9, p < .01.

Discussion

The current study demonstrated that unannounced pill counts conducted by telephone are concordant with unannounced pill counts conducted by home visits, with greater than 90% agreement regardless of clinical level of adherence. Concordance was observed at the level of individual medications as well as at the participant level for both raw pill counts and adherence. The intra-class correlations remained substantial in sensitivity analyses. These findings replicate the initial test of unannounced telephone-based pill count reliability in an independent sample. (11) Importantly, adherence determined by unannounced telephone-based pill counts corresponded with contemporaneous viral loads. The association between telephone-based pill count adherence and viral load was significant and in the order of magnitude observed between unannounced home-based pill counts and viral load. (10)

The unannounced telephone-based pill count was not, however, perfectly concordant with home visit pill counts for 40% of participants. With rare exception the discrepancies in pill counts were within a five pills and in no case did the discrepancies, even when sizable, significantly alter adherence values. The four large pill discrepancies occurred when whole bottles of medications or pillboxes were found in the home that went uncounted on the phone. Our telephone-based pill count protocol includes a system for detecting missed medication bottles by tracking prescription numbers, refill dates, and number of refills remaining. Large errors resulting from pill bottles unidentified over the telephone are therefore correctable when they occur in practice. We also observed that smaller errors were related to participants having more difficulty managing the pill count procedure. These errors may be minimized with additional participant training and corrective feedback. Finally, we observed that pill counting errors were related to participants using pillboxes and taking doses with them when they leave home, a finding that has been previously reported. (11) Assuring that all pill compartments and containers are present and opened during the telephone-based pill count will minimize these errors.

One potential limitation of the telephone-based pill counting in general is its reliance on patients to count their own pills. We did not find an association between discrepant pill counts and education, health literacy, or any other patient characteristic. Also, no participants demonstrated an inability to count their pills or required assistance to complete their pill count during the home visits. Nevertheless, we acknowledge the possibility for patients misidentifying pills and other errors when patients count their own pills. Another potential limitation of telephone-based pill counts is that patients may adjust the numbers of pills counted to conceal missed doses. However, we have no evidence to suggest that participants attempt to guess the numbers of pills left from missed doses or any other mental calculation required for adjusting pill counts. It is also possible that the pill count training and unannounced pill counts themselves influence adherence, suggesting that the average 84% adherence observed in the study is an upper-bound estimate. A limitation of our current study was its use of a convenience sample of HIV positive persons who were taking a variety of ARV medication regimens. Finally, although our sample size was sufficient for the analyses we conducted, future research on unannounced telephone-based pill counts with larger samples will allow for more detailed examination of subgroups, such as gender or age groups. With these limitations acknowledged, we believe that the current study supports the use of unannounced telephone-based pill counts for assessing and monitoring ARV adherence.

We conclude that unannounced home-based pill counts remain among the most reliable and valid methods for assessing and monitoring ARV adherence. Given logistic and cost constraints, however, unannounced telephone-based pill counts offer a viable alternative method of tracking adherence. Unannounced telephone-based pill counts can be performed with patients residing in different cities, states, or even different countries from the person conducting the pill count. The method is particularly useful with patients in rural settings. Certain patients may require adaptation of the pill count procedure, such as providing cell phones to the homeless. Patients who remove their pills from bottles, such as dumping them in bags or other containers, may require collecting pharmacy information to track medications dispensed. Finally, patients who are severely cognitively challenged and simply cannot count or keep track of their counting will require home visits for pill counts. An open question is whether unannounced telephone-based pill counts can be used in clinical care settings for monitoring patient adherence. Research is needed to examine this important application of this new methodology.

Table 5
Characteristics of participants with concordant and discordant telephone and home-based pill counts (N = 68).

Acknowledgements

The authors thank the AIDS Survival Project of Atlanta for their assistance with data collection. National Institute of Mental Health (NIMH) grant R01-MH71164 supported this research. Raymond F. Schinazi is funded in part by the NIH-sponsored Emory University Center for AIDS Research grant 5P30-AI-50409 and the Department Veterans Affairs.

Appendix

Matrix for tabulating antiretroviral regimen pill count.

ARV
Drug
Number
of doses
per day
Pills taken
per dose
Previous
number of
pills
counted
Pills
dispensed
since
previous
pill count
Total pills
counted
Stop dateStart date
NEW ARV’S Started
STOPPED ARV’S ONLY

Prescription bottle information for tracking dispensed medications.

ARVRx numberRx refill date# of refills
remaining
Pharmacy name and
phone #

Contributor Information

Seth C. Kalichman, University of Connecticut.

Christina M. Amaral, University of Connecticut.

Chauncey Cherry, University of Connecticut.

Jodi Flanagan, University of Connecticut.

Howard Pope, University of Connecticut.

Lisa Eaton, University of Connecticut.

Denise White, University of Connecticut.

Moira O. Kalichman, University of Connecticut.

Demetria Cain, University of Connecticut.

Mervi Detorio, Emory University School of Medicine and Veterans Affairs Medical Center.

Angela Caliendo, Emory University School of Medicine and Veterans Affairs Medical Center.

Raymond F. Schinazi, Emory University School of Medicine and Veterans Affairs Medical Center.

References

1. Bangsberg DR, Kroetz DL, Deeks SG. Adherence-resistance relationships to combination HIV antiretroviral therapy. Curr HIV/AIDS Rep. 2007;4:65–72. [PubMed]
2. Kalichman DC, Cain D, Cherry C, Kalichman M, Pope H. Pillboxes and antiretroviral adherence: Prevalence of use, perceived benefits, and implications for electronic medication monitoring devices. AIDS Patient Care STDs. 2005;19:49–55. [PubMed]
3. Petersen ML, Wang Y, van der Laan MJ, Guzman D, Riley E, Bangsberg DR. Pillbox organizers are associated with improved adherence to HIV antiretroviral therapy and viral suppression: a marginal structural model analysis. Clin Infect Dis. 2007;45:908–915. [PMC free article] [PubMed]
4. Simoni J, Kurth AE, Pearson C, Pantalone DW, Merrill J, Frick P. Self-Report Measures of Antiretroviral Therapy Adherence: A Review with Recommendations for HIV Research and Clinical Management. AIDS Behav. 2006;10:227–331. [PMC free article] [PubMed]
5. Wagner G, Miller LG. Is the influence of social desirability on patient's self-reported adherence overrated? J Acquir Immune Defic Syndr. 2004;36:203–204. [PubMed]
6. Bangsberg DR, Hecht FM, Clague H, Charlebois ED, Ciccarone D, Chesney M, Moss A. Provider assessment of adherence to HIV antiretroviral therapy. J Acquir Immune Defic Syndr. 2001;26:435–442. [PubMed]
7. Grossberg R, Gross R. Use of pharmacy refill data as a measure of antiretroviral adherence. Curr HIV/AIDS Rep. 2007;4:187–191. [PubMed]
8. Liu H, Golin CE, Miller LG, Hays RD, Beck CK, Sanandaji S, Christian J, Maldonado T, Duran D, Kaplan AH, Wenger NS. A comparison study of multiple measures of adherence to HIV protease inhibitors. Ann Intern Med. 2001;134(10):968–977. [PubMed]
9. Rabkin JG, Chesney M. Treatment adherence to HIV medications: The Archilles heel of the new therapeutics. In: Ostrow D, Kalichman S, editors. Behavioral and Mental Health Impacts of New HIV Therapies. New York: Plenum Press; 1999.
10. Giordano TP, Guzman D, Clark R, Charlebois ED, Bangsberg D. Measuring adherence to antiretroviral therapy in a diverse population using a visual analogue scale. HIV Clin Trials. 2004;5:74–79. [PubMed]
11. Kalichman SC, Amaral CM, Stearns H, White D, Flanagan J, Pope H, Cherry C, Cain D, Eaton L, Kalichman MO. Adherence to antiretroviral therapy assessed by unannounced pill counts conducted by telephone. J Gen Intern Med. 2007;22:1003–1006. [PMC free article] [PubMed]
12. Bangsberg DR, Hecht FM, Charlebois ED, Chesney M, Moss A. Comparing objective measures of adherence to HIV antiretroviral therapy: Electronic medication monitors and unannounced pill counts. AIDS Behav. 2001;5:275–281.
13. Bangsberg DR, Hecht FM, Charlebois ED, Zopola AR, Holodniy M, Sheiner L, Bamberger JD, Chesney M, Moss A. Adherence to protease inhibitors, HIV-1 viral load, and development of drug resistance in an indigent population. AIDS. 2000;14:357–366. [PubMed]
14. Berg CJ, Michelson SE, Safren SA. Behavioral aspects of HIV care: adherence, depression, substance use, and HIV-transmission behaviors. Infect Dis Clin North Am. 2007;21:181–200. [PubMed]
15. Baker DW, Parker RM, Williams MV, Clark WS. Health literacy and the risk of hospital admission. J Gen Int Med. 1998;13:791–798. [PMC free article] [PubMed]
16. Babor TF, de la Fuente JR, Saunders J, Grant M. The Alcohol Use Disorders Identification Test: Guidelines for use in primary health care. Geneva: World Health Care Organization; 1992.
17. Beck AT, Steer RA. BDI: Beck Depression Inventory manual. New York: Psychological Corporation; 1983.
18. Kalichman SC, Rompa D, Cage M. Distinguishing between overlapping somatic symptoms of depression and HIV disease in people living with HIV/AIDS. J Nervous Ment Dis. 2000;188:662–670. [PubMed]
19. Paterson DL, Swindells S, Mohr J, Brester M, Vergis EN, Squier C. Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann of Int Med. 2000;133:21–30. [PubMed]