|Home | About | Journals | Submit | Contact Us | Français|
The objective was to assess study retention and attendance for two recruitment waves of participants in the Women's Interagency HIV Study (WIHS).
The WIHS, a prospective study at six clinical centers in the United States, has experienced two phases of participant recruitment. In phase one, women were screened and enrolled at the same time, and in phase two, women were screened and enrolled at separate visits. Compliance with study follow-up was evaluated by examining semiannual study retention and visit attendance.
After 10 study visits, the retention rate in the original recruits (enrolled in 1994–1995) was 83% for the HIV-infected women and 69% for the HIV-uninfected women compared with 86% and 86%, respectively, in the new recruits (enrolled in 2001–2002). In logistic regression analysis of the HIV-infected women, factors associated with early (visits 2 and 3) nonattendance were temporary housing, moderate alcohol consumption, use of crack/cocaine/heroin, having a primary care provider, WIHS site of enrollment, lower CD4 cell count, and higher viral load. Among HIV-uninfected women, the factors associated with early nonattendance were recruitment into the original cohort, household income ≥$12,000per year, temporary housing, unemployment, use of crack/cocaine/heroin, and WIHS site of enrollment. Factors associated with nonattendance at later visits (7–10) among HIV-infected participants were younger age, white race, not having a primary care provider, not having health insurance, WIHS site of enrollment, higher viral load, and nonattendance at a previous visit. In HIV-uninfected participants, younger age, white race, WIHS site of enrollment, and nonattendance at a previous visit were significantly associated with nonattendance at later visits.
Preventing early loss to follow-up resulted in better study retention early, but late loss to follow-up may require different retention strategies.
The maturation of the HIV epidemic in the United States continues to impact women and people of color. Although the number of newly reported AIDS cases has decreased in the United States as a result of highly active antiretroviral therapy (HAART) use, in 2005, the proportion of cases reported in women increased slightly to 27%.1 African American women accounted for 61% and Hispanic women accounted for 16% of all reported HIV/AIDS cases in women;1 these two groups combined represent only 22% of the general population of U.S. women.2 Of the estimated 127,150 female adults and adolescents living with HIV/AIDS, 26% had been exposed through injection drug use.1 In 2006, the estimated incidence of HIV infection among U.S. women ranged from 13,400 to 15,000, 24%–27% of all new HIV diagnoses.3
Inclusion of women and people of color in research studies of HIV and AIDS is paramount for the study results to be scientifically valid and has been mandated by the National Institutes of Health (NIH).4 Many individuals, including injection drug users, are excluded from study participation on the basis of poor health status and presence of comorbid conditions.5,6 Additional reasons for not including these groups in research studies vary and may be due to restrictive inclusion and exclusion criteria, reluctance on the part of researchers to enroll individuals that they believe may be hard to recruit or retain, or hesitancy or lack of trust on the part of potential participants.7–9
Recruiting a representative study sample with similar characteristics to the target population is just one aspect of a successful prospective study. Another crucial component is the ability to retain enrolled participants, so that results are not biased as a result of differences between those who are lost and those who continue to participate. Identifying the characteristics of research participants who are more likely to drop out can be valuable in designing methods to reduce further loss to follow-up. In addition, evaluation of when loss to follow-up is more likely to occur can inform and improve the methods for future recruitment of participants.
Our objective was to assess attendance and retention in two recruitment waves of HIV-infected and at-risk HIV-uninfected women into a multisite prospective study in the United States. The hypothesis was that lessons learned from the initial recruitment phase of the Women's Interagency HIV Study (WIHS) led to improved methods for recruitment and retention of women in the second recruitment phase.
During the first WIHS recruitment phase in 1994–1995, the goal was to recruit a sample that was representative of women with HIV/AIDS in the United States. Women were enrolled into one of two groups: HIV-infected or HIV-uninfected and at-risk for acquisition of HIV infection. Almost all HIV-infected women were naive to HAART, and many had already been diagnosed with an AIDS-defining illness. The HIV-infected and HIV-uninfected women had similar demographic backgrounds and a comparable distribution of risk factors for acquisition of HIV infection. A detailed description of the original recruitment showed how similar the women in the WIHS cohort were to the U.S. HIV-infected population.10
From 1994 to 1995, 2054 HIV-infected and 569 HIV-uninfected women were enrolled into the WIHS at six locations within the United States: New York City (two sites), the Washington, DC, metropolitan area, Chicago, southern California, and northern California. Every 6 months, WIHS participants were interviewed using a structured questionnaire and received a physical and gynecological examination. Multiple gynecological and blood specimens were collected at each visit. Follow-up data for the first 24 study visits, through March 31, 2007, were included in the analyses.
For the second wave of recruitment into the WIHS, during 2001 to 2002, the goal was to recruit HIV-infected women with a younger median age and no previous AIDS diagnosis and to identify two groups related to treatment exposure: those who had been on HAART and those who had never been exposed to HAART.11 A comparison group of HIV-uninfected women of similar ages and backgrounds was also enrolled. The second recruitment phase occurred during WIHS visits 15 and 16, and after the second recruitment phase, 737 HIV-infected and 406 HIV-uninfected women were enrolled in the same six WIHS locations. The semiannual study visits for the new enrollees were essentially the same as for the original cohort participants. Follow-up data for the first 10 study visits (5 years) for the new cohort members, through March 31, 2007, were included in the analyses.
Recruitment was performed at a variety of venues for both phases. Nationally, these included HIV primary care clinics, hospital-based programs, research programs, community outreach sites, women's support groups, drug rehabilitation programs, HIV testing sites, and referrals from enrolled participants. Site-specific recruitment sources differed, however, with some sites (Brooklyn and Chicago) recruiting more heavily from primary care clinics and existing research studies.
Recruitment strategies for the original WIHS cohort have been described previously.10–12 We knew from retention analyses of the original cohort that most of the loss to follow-up occurred early, shortly after the first visit. So, for the second wave of recruitment into the WIHS, a two-step screening and enrollment process was used for both the HIV-infected and HIV-uninfected women. At the screening visit, women were asked questions to determine eligibility, and blood was drawn for HIV antibody testing. Once the results of the HIV antibody test were available and if the woman met the inclusion criteria, she was invited back for enrollment into the new WIHS cohort. The HIV-infected women gave written permission for medical record abstraction to determine if they were AIDS-free, had initiated HAART, and if on HAART, to abstract the date of HAART initiation as well as the CD4+ cell count and HIV RNA viral load just prior to HAART initiation.11 Under certain circumstances, the study personnel had the discretion of combining the screening and recruitment process into one visit.
Retention strategies for the original cohort have been described.12 In brief, retention was continually monitored at both the national and local levels. Sites frequently revisited and revised retention strategies to make them more responsive to emerging visit attendance barriers and to address changing study protocols.
Women who did not have a 6-month interview and were not known to be deceased were considered as having missed their visit. Women who moved and transferred to another WIHS site were considered to be a participant of the new site. Women who asked to be disenrolled from the study were considered lost (not retained) for all future visits. For women who were too ill to be seen for a full study visit, those in detention, and those who moved out of the study area, an abbreviated visit (usually over the telephone) was offered.
The National Death Index was searched annually for information on women not seen in the past 12 months. At some sites, local death registries, the Social Security Death Index, and calls to primary contacts were methods used to check for new death information. National death registry data was available through 2006, and local data were available through 2007.
Consent materials were reviewed and approved by the institutional review boards at each of the collaborating institutions, and informed consent was obtained from the participants.
The independent variables of interest were recruitment cohort (new vs. original), age (in decades), race/ethnicity, education, household income, employment, housing, depressive symptoms (Center for Epidemiologic Studies Depression Scale [CES-D] score ≥16), cigarette smoking, alcohol use, history of injection drug use, primary care provider, healthcare insurance, crack/cocaine/heroin use, and WIHS site of enrollment. Logistic regression analyses were conducted using models with all study participants, with HIV-infected only, and with HIV-uninfected only. HIV status was included as an additional predictor in the model that included data from all women. Use of HAART, HIV RNA, and CD4 cell count were examined in the model limited to data from the HIV-infected women. Log10 was used to estimate the effect of a 10-fold change in HIV RNA, and log2 was used to estimate the effect of a 2-fold change in CD4 cell count on the outcome measure. Independent variables that changed over time were included as time varying based on data from the last completed interview.
We assessed compliance with study follow-up in two ways, by examining study retention and by evaluating semiannual visit attendance. For the retention analyses, the dependent variable was attrition, defined as having missed a study visit for more than 12 months (two successive visits) or, if deceased, for more than 12 months prior to the date of death. Retention rates were determined for each WIHS visit cycle (visits 2–24 for the original cohort and visits 2–10 for the new cohort), stratified by HIV serostatus. The retention rate was calculated as the number of women interviewed (either in person or over the phone) in the last 12 months, divided by the number of women enrolled at visit 1 minus those women who have died. Therefore, retention rates can increase from one visit to the next if either the number of women interviewed in the last 12 months has increased or the number of women who have died has increased.
For the attendance analyses, each participant study visit was defined as either attended (seen in person) or missed and analyzed as a dichotomous outcome variable. Women who died were included in the analyses up until the visit when they were reported as deceased. To compare study visit attendance in the original and new cohorts and evaluate the association between missed visits and demographic, behavioral, and clinical variables, we used repeated measures, random effects logistic regression models to account for multiple visits per woman and predictors that could differ for the same woman at different visits. A random subject effect was included to account for the dependence of multiple observations from individual participants.13
To assess our improved retention strategies for the new cohort, one of our goals was to evaluate predictors of participant attendance at earlier visits compared with later visits in the original and new cohorts. We first performed logistic regression using only study visits 2 and 3. To examine whether a previous missed visit affected attendance at a current visit, we included a predictor variable in the model that defined three categories of current and prior visit attendance (attendance at visit 3 given that visit 2 was attended, attendance at visit 3 given that visit 2 was missed, and attendance at visit 2 for which all cases attended visit 1 [the reference]). We then performed logistic regression using visits 7–10 and restricted the analyses to women who were last seen at either visit 5 or visit 6. In these analyses, we evaluated the association between attendance at two previous visits and attendance at a current visit by including a three-category predictor variable (attended previous visit but missed the visit before previous, missed previous visit but attended the visit before previous visit, and attended both previous and before previous visits [the reference]). For the missed visits, we used predictors from the previous attended visit. We excluded from the analysis visits where both previous and before previous visits were missed to avoid using predictors that were less current.
Among the 975 HIV-uninfected women enrolled at baseline, 19 women seroconverted for HIV during follow-up. Because of the small sample size, these 19 women were excluded from all analyses. Statistical analyses were performed using SAS® software version 9.1 (SAS Institute, Cary, NC) and S-PLUS® software version 8.0 (Insightful Corporation, Seattle, WA).
A total of 2623 women were recruited into the original cohort study (2054 HIV-infected and 569 HIV-uninfected women), and 1143 women were recruited into the new cohort study (737 HIV-infected and 406 HIV-uninfected women). Eighty percent of the original participants attended their second WIHS visit compared with 91% of the new participants (p<0.001). After the first 10 visits, 480 of the 3766 women had died (463 HIV-infected and 17 HIV-uninfected), and 1251 HIV-infected women (45%) reported having an AIDS-related illness.
Comparisons of baseline characteristics for the original and new participants are summarized in Table 1. Compared to the original recruits, women recruited in 2001–2002 were significantly more likely to be HIV-uninfected, younger, Latina, have higher household incomes, be employed, and report no identifiable risk for HIV infection. They were also significantly less likely to have depressive symptoms, be a current smoker, drink alcohol, have a history of injecting drugs, or have a primary care provider. Among HIV-infected women, women in the new cohort had a higher mean CD4 cell count and lower median HIV viral load and were more likely to be HAART users.
The retention rates by visit number for the original and new cohort participants are shown in Figure 1. Among the original cohort participants, there was a sharp decline in retention rates between visits 1 and 3 (89% and 83% retained at visit 3 among HIV-infected and HIV-uninfected women, respectively), and retention rates were significantly higher (p<0.001) for HIV-infected women (83% at visit 10) compared with HIV-uninfected women (69% at visit 10). Among the new cohort participants, there was only a modest decline between their first and third visits (99% retained at visit 3 among HIV-infected women and 96% retained among HIV-uninfected women). The retention rates at visit 10 were 86% for both HIV-infected and HIV-uninfected women.
Figure 2 superimposes the retention rates for the new cohort on the same temporal scale as the original cohort and visually demonstrates the improved retention rates of the new cohort compared with the original cohort, especially for the HIV-uninfected women. Among the HIV-uninfected women, the retention rates for the new cohort were statistically higher than for the original cohort for visits 3–10 (p<0.001 for each visit). Among the HIV-infected women, the retention rates for the new cohort were statistically higher than for the original cohort for visit 3 (p<0.001), visit 4 (p=0.015), visit 5 (p=0.040), and visit 9 (p=0.039).
The factors associated with nonattendance at WIHS visits 2 and 3 are shown in Table 2. Among all study participants, the following factors were significantly associated with nonattendance: being HIV-uninfected (OR=1.98), recruitment into the new cohort (OR=0.20), younger age (OR=0.71), temporary housing (OR=4.16), employment (OR=0.59), having depressive symptoms (OR=1.47), moderate alcohol consumption (OR=1.53), use of crack/cocaine/heroin (OR=3.77), having a primary care provider (OR=2.37), and WIHS site of enrollment (OR range 0.37–3.47). Among HIV-infected women, the factors associated with nonattendance at WIHS visits 2 and 3 were temporary housing (OR=2.80), moderate alcohol consumption (OR=1.46), use of crack/cocaine/heroin (OR=2.56), having a primary care provider (OR=2.14), WIHS site of enrollment (OR range 0.33–2.17), lower CD4 cell count (OR=0.84), and higher viral load (OR=1.37). Among HIV-uninfected women, the factors associated with nonattendance at WIHS visits 2 and 3 were recruitment into the new cohort (OR=0.15), household income <$12,000per year (OR=0.36), temporary housing (OR=4.81), employment (OR=0.37), use of crack/cocaine/heroin (OR=3.06), and WIHS site of enrollment (OR range 0.87–3.01).
In analyses evaluating the factors associated with nonattendance at WIHS visits 7–10 among all study participants, the following predictors were significant: younger age (OR=0.76), white race (OR=1.70), employment (OR=0.82), having a primary care provider (OR=0.77), having health insurance (OR=0.76), WIHS site of enrollment (OR range 0.53–1.89), and nonattendance at a previous visit (OR range 4.91–34.12) (Table 3). In multivariable analysis of HIV-infected study participants, younger age (OR=0.78), white race (OR=1.58), having a primary care provider (OR=0.77), having health insurance (OR=0.74), WIHS site of enrollment (OR range 0.53–1.93), higher viral load (OR=1.16), and nonattendance at a previous visit (OR range 3.31–30.7) were significantly associated with nonattendance at a later visit. In adjusted analysis of HIV-uninfected study participants, younger age (OR=0.77), White race (OR=2.37), WIHS site of enrollment (OR range 0.46 to 2.07), and non-attendance at a previous visit (OR range 10.5 to 41.7) were significantly associated with non-attendance at a later visit.
Fourteen years after the first study participant was enrolled, the WIHS has remained successful at retaining women study participants who were representative of the communities affected by HIV and AIDS in the United States. After 10 study visits (5 years), the overall retention rate in the new WIHS cohort was 86% for both the HIV-uninfected and HIV-infected women. In the original cohort, after 24 study visits (12 years), the retention rate was 75% for the HIV-infected women and 62% for the HIV-uninfected women. This is a remarkable achievement, considering that when women were originally recruited, they likely had no idea that the study follow-up would continue this long.
The women recruited in the more recent years, 2001–2002, had better overall study retention and significantly better attendance at the early visits (through visit 3), even after adjusting for potentially confounding covariates. However, the early benefit of the two-step recruitment and enrollment strategy that was adopted for the second wave of recruitment did not result in better attendance at the time of the later visits (7–10). Besides the two-step recruitment, there likely are other unmeasured factors that may have contributed to improved attendance at the early visits, including study staff members who were more experienced and adept than they were in the first years of the study at tracking, locating, and retaining study participants.
Interestingly, among all participants, HIV serostatus was a factor in study attendance only for the early visits, with HIV-uninfected women less likely to return for follow-up visits 2 and 3. After this early loss, attendance for later visits was similar for the HIV-infected and HIV-uninfected women. Similarly, living in temporary housing was a very strong predictor of nonattendance at the early visits, but after this early loss to follow-up, attendance at the later visits did not differ by housing status. Other studies and our previous analysis also found unstable housing to be a predictor of loss to follow-up,12,14,15 but these studies did not differentiate early loss from late loss.
Similar to other prospective studies of HIV infection14,16–19 and consistent with our previous results,12 we found that retention was lowest among younger participants. Younger participants tended to be more geographically mobile, had more child bearing and rearing responsibilities (OR=0.74, 95% CI 0.68-0.81per decade of age), and may perceive less benefit from participation in an ongoing cohort study than older participants.
The WIHS site of enrollment was also a strong predictor of both early and late attendance. As we previously reported, this is most likely due to where women were recruited (clinic patients vs. street outreach) and whether or not clinical care was provided at the site of enrollment.12 Among women in the original cohort, site of enrollment was significantly associated with having a primary care provider (p<0.01). Although there were great efforts made to have standardized quality assurance procedures and protocols and a high degree of similarity in the staffing and manner in which participants were treated, retention and attendance rates differed by site of enrollment.
We also found that attendance at the later visits was lower among white women in the WIHS than among women in other racial/ethnic groups; this was true for both the HIV-infected and HIV-uninfected women and after adjusting for potential confounders. This finding is similar to results from other HIV studies that included women.14,18 It is not clear why there is a difference in attendance by race/ethnicity; some possible explanations include a higher perceived benefit for continued participation among women of color compared with white women, a better relationship with study staff, and a stronger desire to give back to the community.9,20
Among HIV-uninfected women, a consistent risk factor for both early and late nonattendance was being unemployed. Even though unemployment may be associated with lower educational attainment and substance use, in analyses that controlled for these factors, employment was still a significant predictor.
We found attendance by HIV-infected women to be better among those who were healthier (higher CD4 cell counts and lower HIV viral load at last study visit), which is similar to reports from other studies of people with HIV infection.16,19 Curiously, however, attendance at the early visits was better for women without a primary care provider, and attendance at a later visit was better for those with a primary care provider or health insurance. This might be because initially women without a primary care provider were more motivated to participate (and receive some healthcare through the study protocol) than were women who already had a primary care provider. Over time, however, women who had a primary care provider might be more adherent with care and also more adherent to our study protocol.
Using repeated measures random effects logistic regression models, this is among the first prospective studies to evaluate predictors of study attendance both early and later during follow-up. Our analytical methods allowed for the predictors to both vary over time and be current. For example, in our analysis of attendance at visits 7–10, rather than using baseline measures, we used those recorded at the most recent visit and excluded women who were not seen after visit 5. This prevented the predictors from becoming stale and allowed us to compare the factors associated with the early loss group with those of the late loss group. In addition, by measuring attendance at each visit, we were able to demonstrate that women who miss visits, especially consecutive visits, are more likely to miss future visits. Thus, efforts should be made to bring women back for a follow-up visit after their first missed visit.
A limitation of our analysis is the smaller sample size of HIV-uninfected women, who represented 22% of the entire cohort. Additionally, some of the statistically significant predictors of study attendance, such as race/ethnicity, are probably surrogates for other unmeasured factors, such as perceived benefit of study participation.
Lessons learned about recruitment and retention of the original WIHS cohort participants were extremely beneficial at improving short-term retention of the new WIHS cohort members and short-term and long-term retention of the HIV-uninfected women in the new cohort. Tailoring recruitment and retention strategies based on the individual and dynamic characteristics of targeted and enrolled cohort participants is an important approach to ensuring long-term retention of underrepresented populations in research studies. Study staff members must remain vigilant for emerging barriers to recruitment or visit attendance and modify study protocols and procedures accordingly and in a timely manner. Prospective studies that enroll active drug users and those with unstable housing should make a special early effort to keep these participants in follow-up, as they are at high risk for early study attrition. The WIHS continues to demonstrate that women of color and those living on the margins of society are capable of contributing valuable data to long-term observational studies. Without this representation of the community members most affected by the HIV/AIDS epidemic, it would be impossible to generalize the findings back to the community of interest and address rapidly evolving scientific priorities among an aging HIV-infected population.
Data in this article were collected by the Women's Interagency HIV Study (WIHS) Collaborative Study Group with centers (Principal Investigators) at New York City/Bronx Consortium (Kathryn Anastos); Brooklyn, NY (Howard Minkoff); Washington, DC, Metropolitan Consortium (Mary Young); The Connie Wofsy Study Consortium of Northern California (Ruth Greenblatt); Los Angeles County/Southern California Consortium (Alexandra Levine); Chicago Consortium (Mardge Cohen); Data Coordinating Center (Stephen Gange). The WIHS is funded by the National Institute of Allergy and Infectious Diseases (UO1-AI-35004, UO1-AI-31834, UO1-AI-34994, UO1-AI-34989, UO1-AI-34993, and UO1-AI-42590) and by the National Institute of Child Health and Human Development (UO1-HD-32632). The study is cofunded by the National Cancer Institute, the National Institute on Drug Abuse, and the National Institute on Deafness and Other Communication Disorders. Funding is also provided by the National Center for Research Resources (UCSF-CTSI grant number UL1 RR024131). The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of theNIH.
We are deeply grateful to the women who consented to be part of this study. We are also grateful to Dr. Peter Bacchetti for his statistical guidance of the analyses, and we thank Michael Schneider for assistance in preparing the data and for programming and Heneliaka Jones for help with manuscript preparation.
The authors have no conflicts of interest to report.