|Home | About | Journals | Submit | Contact Us | Français|
The conduct of longitudinal clinical trials must involve effective strategies to retain study participants in order to ensure internal validity, adequate statistical power and generalizability of results.
In a large trial in Peru, we implemented various retention strategies to maintain high participation rates over time.
Novel participant retention strategies were used to follow highly marginalized populations for two years because traditional locator information, such as telephone numbers and official identification (eg, passport, driver's license, the local equivalent of a social security number) were often unreliable or unavailable. These strategies included detailed preliminary ethnographic research to identify the behaviours of key target groups, approaches to develop strong informal bonds between project staff and participants outside of study settings, and methods to enhance positive participant attitudes towards the study.
The overall study retention rate after two years was 84%, even though only 26% of the study populations supplied complete locator information (telephone, address and the names of two friends).
The retention strategies used were labour intensive and iterative, which could prove difficult to replicate.
The two-year retention rate in this study was sufficient to maintain required sample sizes. The methods used to maintain contact with the populations were labour intensive, low tech and adequate for these populations and could be used to retain study participants in other marginalized, urban, low-income areas.
Participant attrition must be minimized in longitudinal research trials to maintain internal and external validity, and statistical power [1–3]. A recent analysis of methods for individual- and group-level HIV intervention research rated attrition among the top seven threats to internal validity . For this reason, most research studies utilize detailed strategies for retaining participants. Although some reports from the field describe practical guidelines for reaching and retaining the hardest to reach populations, relatively little has been published on the success of different cohort retention methods [3,5], especially in developing country settings.
Recently, high retention rates have been reported in retaining “hidden” or socially marginalized groups (ie, people who are especially difficult to recruit and retain such as drug users, certain sexual and ethnic minorities, the mentally ill, homeless individuals, low-income, literacy and education), with follow-up rates of 90% or greater [4,6,7]. These studies show that successful retention involves time, resources, cultural sensitivity, and, above all, the collection of as much locator information on individuals as possible [4,7,8]. Additionally, retention efforts in studies of young people have reported successes using incentives, flexibility in participant visits, and frequent contact with participants [5,9]. However, the majority of these findings have been reported in observational not intervention-based studies .
One recent review of retention procedures used in studies of behavioral interventions in cohorts reported on several non-locator based retention methods that were qualitatively described by study staff as effective in retaining participants . These include describing the benefits of their participation to study subjects, flexibility on the part of the researcher regarding time or place of study visit, and minimizing the burden on the participant, for example, by decreasing the duration of the study visit. The field is just beginning to evaluate the effectiveness of such methods of maintaining a cohort, and few assessements of their use or utility have appeared. Additionally, some of the procedures that have been described would not be amenable to a formal clinical trial, where maintaining the same standard study procedures for each individual subject is important. In developing country settings there is also concern about providing overly large incentives for participation, since these could induce participation by individuals who would not otherwise choose to do so.
Having a good tracking system, including complete locator information, is still paramount in retaining participants. Senturia et al. demonstrated that the proportion of participants with complete follow-up information was positively associated with the number of contact names the index participant provided . This indicates the importance of not only collecting participants' names, telephone numbers and home addresses, but also the necessity of collecting contact information of participants' friends or family members. Complete locator information, however, is not necessarily available when conducting research with young people in developing country contexts, since many people in such locales do not have telephones or receive mail, and can be quite distrustful of providing such information.
While collecting detailed locator information may represent an ideal, contact information such as telephone number, e-mail address and social security number often do not exist or can be unreliable in resource-poor areas and among marginalized populations. This was the issue faced in the current study (described more fully below), which we conducted in Lima, Peru where high participant retention was crucial but many of the types of locator information described earlier were not available. For example, participants in our study did not own cars and often did not have drivers' licenses, telephone numbers or social security numbers. When we began the research in the late 1990s, we faced the challenge of tracking young, socially marginalized, sexually active populations in resource poor settings who had not been studied previously and about whom little was known regarding best practices for study retention. We needed novel, customized approaches to retain study participants – methods that depart from those tested in resource-rich settings – in order to attain the high retention rates necessary for valid results.
This article describes a novel and comprehensive approach to retaining a longitudinal, cohort of marginalized populations for a study in Lima, Peru where the standard retention tools traditionally used were unavailable or unreliable. The cohort retention approach described here is from the Peru site of the US National Institutes of Mental Health (NIMH) Collaborative HIV/STD Prevention Trial, in which the efficacy of a community-based two-year HIV/STI prevention intervention is being tested via a randomized, controlled trial, using biological and behavioral markers as end points. The biological and behavioral markers are being collected at baseline and annually until the end of the study. Peru is one of the five sites in the larger study. The intervention uses the “Popular Opinion Leader” (POL) intervention  that was originally developed and tested in the United States with gay/bisexual men in bars in small cities, but has been adapted for young people at risk in developing countries. In Peru, a cohort of young people from poor neighbourhoods or barrios is being followed for two years to evaluate the efficacy of the POL intervention. In this article, we report on the retention of the Peru cohort included in the NIMH study (the 24 months of HIV/STI surveillance).
The trial is being conducted in barrios in and around three coastal cities in Peru: Lima, Trujillo and Chiclayo under the direction of Cayetano Heredia University in Lima, in conjunction with the University of California, Los Angeles and the University of California, San Francisco. The goal of the trial is to test the efficacy of an HIV/STD prevention intervention, POL, which is based on diffusion of innovations . In this intervention approach, well-respected and liked opinion leaders in the community are mobilized to talk with their peers about HIV/STI risk reduction and the need to obtain testing for HIV/STI. The 15% of the populations who are most credible and well-liked are recruited into the intervention and are trained about how to talk with their friends about HIV/STI in the context of casual conversations. Trainings occur over a four-week period, and reunions, to support the C-POLS (the community popular opinion leaders) are held every six weeks. This intervention approach is being tested via a randomized, controlled trial where the barrios are the units of randomization. The endpoints of interest are HIV/STD incidence and sexual risk behavior, specifically the reduction of unprotected casual sex. This report focuses on the retention methods used in the eight barrios in and around Lima, half of which were randomized to intervention and half to comparison conditions, since all study procedures are completed in this city. The study is continuing at this time in the two northern cities.
The Peru study protocol and consent procedures were approved by five IRBs at the following institutions: Cayetano Heredia University, Lima, Peru; University of California, Los Angeles; University of California, San Francisco; Research Triangle Institute, Research Triangle Park, North Carolina; and the Naval Medical Research Center, Bethesda, Maryland. All study participants provided voluntary, written, informed consent prior to study participation. The informed consent included information on the assessment and tracking visits, the contact information requested throughout the study (indicating that provision was not necessary for participation) and the activities undertaken by the study staff to maintain contact with participants.
Extensive formative research was conducted at the outset of this project. Importantly, we needed to tailor POL to meet the cultural characteristics unique to the target populations in Peru . The study included a formative period that utilized ethnographic research, social mapping, and social networking to collect information on community members' beliefs, attitudes, and behaviors regarding HIV and STIs, which led to the identification of three target populations at particularly high risk for HIV/STIs. In addition, the formative research helped to identify venues where the three study populations congregated within the barrios (eg, hair salons, street corners, soccer fields, bars). The formative phase of the study also revealed that potential participants did not have access to social services (and thus did not have national identification numbers, the equivalence of social security numbers), were often not able to obtain identity cards, did not have cars, and usually did not have a home telephone (nor did their friends and family). Additionally, the three populations (described below) were often mistrustful of all outsiders to their barrios, including study staff. Once research activities began in the barrios (including the formative research), many people needed to be reassured, repeatedly, that the study staff was not going to abandon working with them and needed and valued their participation. Very few of the individuals in these three populations had ever been involved in research before. These results informed the tracking and retention procedures implemented in the trial.
The trial was implemented after the formative research was conducted. This involved recruiting and maintaining a cohort comprised of the three study populations, and included annual visits to the barrios to assess biological and behavioral endpoints. Temporary project offices were set up in the barrios for these visits. Strategies for tracking participants were developed based on information gathered during the formative work in order to maintain high retention rates.
The cohort that was used to assess the efficacy of the intervention was independent of the intervention. The cohort has its own name, the Community Sexual Health Study (ECOSS, its Spanish acronym), logo, and identity, completely apart from those of the intervention and is being followed in both intervention and control barrios. The intervention is called, “Que te Cuentas?” (What's Up?). The cohort was described as a sexual health study to people in the barrios during the formative research, as well as throughout the study.
The formative research revealed three populations that were particularly at risk for HIV/STI. These populations became the focus of the intervention, and hence, were recruited into the cohort.
These are heterosexually-identified young men who are un- or under-employed, have relatively little education, and spend their time in gathering places in their barrio, such as street corners, soccer fields and empty lots.
These men often drink alcohol (beer or inexpensive liquor), smoke marijuana or cocaine base and/or inhale glue as social activities to pass the time. Many are involved in illicit or illegal activities such as petty theft, pick pocketing and drug dealing, and many are involved with gangs. Study data show that esquineros often report unprotected intercourse with women, and some report sexual contact with other men in exchange for gifts or money .
These are men who have sex only with men and do not self-identify as heterosexual but rather as “homosexual”, travesti (transvestite) or as mujeres (women). A few men in this group also self-identify as “gay” in the North American sense, but they are a minority. Groups of MSOM tend to congregate in the barrios at volleyball courts and small, neighborhood hair salons where they may work. Alcohol and marijuana use are common. Transactional sex is frequent and is characterized by the MSOM providing gifts, haircuts or alcohol in exchange for sex with the esquineros. Ethnographic data show that MSOM frequently have concurrent sex partners, and often report sex without condoms, especially during transactional sex.
These are women who spend time on the streets socializing with men, usually with the esquineros and/or the MSOM. They do not form their own social groups (in contrast to the esquineros and the MSOM) but instead are found alone or in pairs among the male groups. Alcohol use is frequent, especially when socializing with esquineros and MSOM. Most movidas are un- or underemployed, have little education and many have children, though most are not legally married. While the majority of the movidas report having a primary partner, sex with other men is not unusual. Condom use is infrequently reported. Some engage in transactional sex.
These three populations are subgroups of the general population in low-income barrios in Peru. A more in-depth description of these population segments is available in a separate publication .
As described below, various types of study staff worked to recruit and follow the study cohort. All study staff received training with a particular focus on the importance of confidentiality. While none of the staff below ever knew cohort participants' survey responses, they also were trained that: 1) they could not disclose to anyone in the barrios who was or was not in the cohort, and 2) they should completely avoid talking about any topics related to sexual health, safer sex, condoms, alcohol/drug use or any other project related topics. None of the staff described below were involved in any way with the intervention and other than the ethnographers, did not know about the intervention's connection to the cohort.
Trained, bachelor's level social scientists worked during the formative ethnographic phase and later conducted tracking activities. The Ethnographers reported directly to the study's Retention Coordinator and acted as supervisors for Project Promoter and Helper Participants. The Ethnographers were the only project staff to record locator information on participants, which was then kept under lock and key in the study offices to protect participant confidentiality. There was one ethnographer per barrio on average.
These were young, often gay men, with similar backgrounds to those in the study populations. Many had served as key informants during the ethnographic stage and therefore knew the populations well. They often lived in the barrios and were trained by project staff to conduct field visits in study venues to maintain a presence in the community. Their training included how to communicate and work with the target populations, how to communicate empathy, and the importance of maintaining the confidentiality of the participants. Project Promoters reported to the Ethnographers and the study's Retention Coordinator. On average there was one Project Promoter per barrio.
These were cohort participants who assisted staff with retention efforts by spreading the word to others in the barrios when assessment visits were going to take place. To assure confidentially, the helpers did not know who was in the cohort, but simply distributed invitations to others in the barrios to participate in the cohort. For example, Helper Participants would tell people socializing together on a street corner that ECOSS was back and if anyone was a participant that they should go to the study office when possible. Helper Participants reported to the Ethnographers. There was an average of two Helper Participants per barrio.
We utilized the above personnel to implement a variety of retention strategies in this study, some of which were fairly typical and some that were novel. We describe these below.
The following information, if available, was collected on all cohort participants and was updated as indicated for tracking and retention efforts: first and last names; nicknames or aliases; telephone numbers; home addresses; and the names, addresses and telephone numbers of two contact persons (friends, family members, neighbours, etc.). Ability and willingness to provide this information was not required for participation.
These were the visits to the barrios to assess the trial's behavioral and biological end points. Retention rates were calculated based on data collected during these assessments. These assessments include the baseline (month 0), plus two annual follow-up assessments (at months 12 and 24). We collected the following information during these visits: informed consent (at baseline visit only); updated locator information; and epidemiological/biological data and self-reported behavioural information. In addition, we took digital photographs of willing participants for a study ID card and conducted HIV/STI testing and counselling. Compensation was provided to participants for their time and effort (US $4.00). Two weeks later, participants returned for their HIV/STI test results, and received post-test counselling and obtained treatment for any incident STIs. Health and mental health referrals to local health centres were also provided as needed. Participants were compensated an additional US $3.00 at this time. The assessment visits were all conducted at temporary project offices in the barrios.
The tracking visits involved activities that exclusively focused on the collection of information that would help in retaining and tracking the cohort. These visits occurred between the yearly assessment visits, at months four, eight, 16 and 20, and were conducted in the same temporary project offices in the barrios as those used for the assessment visits. Prior to these visits, the Ethnographers and Project Promoters visited the barrios to tell the participants the date and time of the tracking visit. If study participants so desired, they went to project offices so that the Ethnographers could update their locator information and in return they received pamphlets on various health topics unrelated to HIV/STIs and a small token for their study participation (keychain, pens, bottle opener) and US $3.00 compensation.
Participants who attended study visits were asked to tell their friends that the study staff was trying to contact all cohort participants. Although these Helper Participants were never told the identities of other participants, many study participants had friends who were also participating and were able to encourage people who missed study visits to contact study staff.
The informal field visits were primarily social in nature, during which the Ethnographers and Project Promoters visited with study participants in the barrios, engaging in activities such as chatting with men and women at soccer games, at hair salons or at street corners. The goal of these visits was to maintain the study's presence in the community and no compensation was provided. In addition, these visits allowed project staff to monitor the location of each participant and in particular to determine if a participant was still in the barrio or had left for any reason. In this way, an approximation of the number of study participants available for the tracking and assessments visits was known in advance. When participants were not located during these informal monthly visits, more intense retention efforts were indicated. Thus potential participant losses were identified early and work to locate the participant could initiate well before an assessment visit in an effort to preempt potential attrition.
Ethnographers and study promoters were trained to talk to participants as peers, and to avoid a “researcher-research subject” (or “top-down”) dynamic. Staff chatted informally with research participants about life, friends and family, in order to create a friendly, empathic and on-going relationship. This “horizontal treatment”, using staff of similar age and economic background to the study participants, seemed particularly helpful in retaining the participants, particularly given community members' distrust of outsiders.
In addition to the other study visits, periodically the Ethnographers and Promoters also took part in barrio events when invited to do so by participants. These local events included sporting events, barbeques, birthday celebrations, and so forth and demonstrated to participants that the study staff cared about them as more than just study participants.
The cohort had a logo, a visual image that the participants could identify with the study. This logo was used at the local study offices during the assessment visits; additionally the logo was placed on the small tokens given to participants during tracking visits (key chains, pens, bottle openers, and so on). Many of the participants carried these items with them, and thereby had a constant reminder of the cohort. Participants sometimes showed these items to the Ethnographers and Study Promoters to demonstrate their commitment to the study.
Each participant received a study ID card that identified him/her as part of the study for tracking purposes. Participants who agreed to have a digital photograph taken had their photograph included on the ID card. Anecdotally, numerous cohort participants commented that the cards also helped them feel part of something important and prestigious since research projects had only rarely, if ever, been conducted in their barrios before. The ID cards displayed Cayetano Heredia University's logo, lending credibility to the study and the participants. Including a digital photograph of willing participants allowed the verification of participants' identity at study visits and gave the participants a sense of belonging to an “official” activity. Indeed, staff noted that participants treated these cards like a “VIP membership card” or a “Club Card”.
As the availability of inexpensive Internet access in the barrios increased over the duration of the study, study participants began to use Internet chat and e-mail to contact study staff. Tracking staff, likewise, gave out their e-mail addresses and chat IDs. This information was only exchanged at the request of the participants.
Participants provided contact information as follows: 91% and 98% provided their address at baseline and by the second follow-up, respectively (see Table 1). Telephone numbers were provided by 43% of participants at baseline and 48% by the second follow up. The names of two friends who would know a participant's whereabouts were provided by 42% and 55% of participants at baseline and follow-up two, respectively (see Table 1). By the end of the second year of the study, nearly one third of the participants, 328 (26%), had provided all three pieces of contact information (address, telephone number and names of two friends).
A total of 1263 participants were enrolled into the study in Lima and completed the baseline assessment (month 0). The baseline cohort included 878 esquineros, 77 movidas and 308 MSOM. The tracking visits during year one (at four and eight months) had retention rates of 76% (n = 957) and 62% (n = 785) respectively; it is important to remember that it was not imperative for assessment participants to come to tracking visits since no study data were collected at these times. An additional 18% and 32% at four and eight months, respectively, communicated their continued presence in the barrio to study staff even though they did not attend the tracking visit. Therefore, at four months into the first year we were able to account for 93% of the original cohort and at eight months could account for 94% of the cohort by these various means.
At the first follow-up assessment, during month 12, 92% (n = 1166) of the cohort was assessed. At the year two tracking visits, at months 16 and 20, the retention rates were 57% and 62% reporting in person, with and an additional 36% and 29% reporting their presence even though they did not attend the tracking visits. Hence at months 16 and 20 we were able to account for 93% and 91% of the sample, respectively.
At month 24, the final follow-up assessment, participant retention was 84% (n = 1062). At this assessment, the remaining 1062 participants included 738 esquineros, 71 movidas and 253 MSOM. Hence, we had retained 84% of the esquineros, 92% of the movidas and 82% of the MSOM. The 201 participants lost to follow-up included: 19 deaths (HIV related illnesses, accidents, and violence); 81 who moved from the study barrios; 34 who withdrew from the study; 29 who were incarcerated; seven who were in drug rehabilitation centers and 13 who did not have enough time to participate in the study. Of the initial cohort, only 18 participants (1.4%) could not be located at all.
With a retention rate of 84% at two-year post baseline assessment, this study demonstrates successful participant tracking and retention in a resource poor setting with marginalized and often hidden populations. It could be argued that the attrition rate was much lower than 16%, since we had accounted for all but 1.4% of the original cohort. This high retention rate is particularly striking because special technology was not utilized and only very basic locator information was collected. This retention rate was achieved in the absence of standard retention tools (ie, complete locator information and/or technological methods) for over half of the study sample. These results may be attributable in part to the study design, starting with the formative research that helped to identify the study's target populations, which led to a thorough understanding of the three populations' sociocultural environment , and providing insights into potential barriers and facilitators in retaining these populations in a longitudinal cohort. Rather than functioning as a separate process to track participants for the overall study, eg, a procedure followed only when participants missed a visit, this study employed a constant process of remaining in regular contact with participants, both formally and informally, over several years, thereby creating a social and study culture that supported visit attendance.
The key retention strategies discussed (see Table 2) appeared to have enhanced the cohort's connection with the study, as well as reinforcing the relationships between study staff and participants. In retrospect, we recognize the similarity of this approach with McQuiston's recent description of “Natural Helpers” in community-based HIV prevention programmes, where “the recruitment and retention process has shown how knowledge of the target community and cultural values were interwoven into all aspects of program development” (p. 103) ; although, in contrast, their idea was used for retention in a community-based health programme and not a clinical trial. Our staff developed rapport and exhibited concern for the participants' well-being and living conditions, which was not the experience these participants previously had when interacting with mainstream society. Over time, participants began to identify with the study, often using their study ID cards that displayed the University's logo, to show that they were part of something that was important to them, and, that the University considered them to be important.
Since the retention strategies, including supporting cohort participants' positive regard for participating in the cohort, were conducted in both intervention and control barrios, it is not expected that the tracking methods will affect outcome variables differentially. It is also not expected that staff expressing positive regard for the cohort participants would influence the extent to which the latter have unsafe sex since the staff did not discuss sexual behaviour, HIV/AIDS or STI with community members. An abundance of trials over the years have shown that sexual behaviour and disease incidence are not easily modified even through intensive social/behavioural interventions, and it is doubtful that study staff's work to keep the cohort intact would make a difference in sexual risk behaviour. Hence, we do not expect that this amount of interaction between staff and cohort participants would influence the outcome variables for this trial.
The use of the Internet became a valuable retention tool that was used during the final years of the study. Ironically, while a participant may not have had a phone number, s/he was likely by the early 2000s to have access to the Internet since community Internet “cafés” are common and extremely inexpensive even in poor urban areas of Lima (average cost of 1 Nuevo Sol or about US $0.33 per hour). Participants who provided an e-mail address or Internet chat name were able to contact staff or vice versa in order to remain in contact, answer questions or make reminders for upcoming study visits.
It is important to recognize that these study populations are more vulnerable than Lima's general population in terms of risk to health and well-being; indeed, 19 participants died during the trial from accidents, violence or illness and 29 were incarcerated and therefore unavailable for follow-up at the time of the final assessment visit. Public health and biomedical intervention researchers may avoid studying these populations out of concern about the feasibility of retaining participants once they are recruited. Our retention rates show that effective trials with retention rates that maintain the sample size needed for adequate statistical power are possible in marginalized populations in resource-poor settings.
The approaches we used, while technologically simple, are labour intensive. They involve a large number of staff members conducting numerous visits to residential areas over time in order to establish and maintain trust and rapport with the study groups. While this may not be the best approach to use in resource rich settings where labor costs are substantially higher, in Peruvian barrios and in other poor areas where human resources are relatively inexpensive (compared to technology), this approach is surprisingly effective since multiple study staff can be trained and deployed to the study venues to provide a high level of study presence. Our experience demonstrates that “difficult-to-reach” populations can be effectively studied and successfully retained in long-term cohorts when a systematic, anticipatory retention system is fully integrated into the study design. Longitudinal studies working with similar populations in resource poor settings should consider retention plans that harness participants' desire to be included in projects they perceive to be important and to be treated by researchers as peers, and with respect. While retention was not the main focus of this study, we hope the evaluation and dissemination of the results its retention strategies will serve as a model for similar evaluations of cohort retention approaches in clinical trials in resource limited settings with patient populations that cannot be followed through traditional means.
This study was supported by grant U10 MH61536 of the NIH/NIMH, a multisite collaborative HIV/STI prevention trial that was conducted in China, India, Peru, Russia and Zimbabwe.
The opinions and assertions expressed herein are those of the authors and do not necessarily reflect the official policy or position the US Departments of the Navy or Army, of the US Department of Defense, the US Government, or any other organization listed.