PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
AIDS. Author manuscript; available in PMC 2013 June 28.
Published in final edited form as:
PMCID: PMC3695696
NIHMSID: NIHMS307210

Weighing the Gold in the Gold Standard: Challenges in HIV Prevention Research

Abstract

Objective(s)

Few HIV prevention interventions have been evaluated in randomized controlled trials (RCTs). We examined design, implementation, and contextual considerations that may limit detection of a positive or adverse effect in HIV prevention trials.

Design

A systematic review of late phase RCTs for prevention of sexual transmission of HIV that 1) randomly allocated intervention and comparison groups; 2) evaluated interventions to prevent sexual transmission in non-pregnant populations; and 3) reported HIV incidence as the primary or secondary outcome.

Methods

PubMed/MEDLINE, other electronic databases, and electronic conference proceedings of recent HIV/AIDS-related conferences were searched to identify published or unpublished trials meeting the inclusion criteria. Descriptive, methodological, and contextual factors were abstracted from each trial.

Results

The review included 36 HIV prevention RCTs reporting on 38 unique interventions. Only six RCTs, all evaluating biomedical interventions, demonstrated definitive effects on HIV incidence. Five of the six RCTs significantly reduced HIV infection: all three male circumcision trials, one trial of STI treatment and care, and one vaccine trial. One microbicide trial of nonoxynol-9 gel produced adverse results. Lack of statistical power, poor adherence, and diluted versions of the intervention in comparison groups may have been important issues for the other trials that demonstrated “flat” results.

Conclusions

Almost 90% of HIV prevention trials had “flat” results, which may be attributable to trial design and/or implementation. The HIV prevention community must not only examine evidence from significant RCTs, but must also examine flat trials, and address design and implementation issues that limit detection of an effect.

Keywords: HIV infection, prevention, microbicide, circumcision, sexually transmitted infections, adherence

Introduction

The global need for effective HIV prevention programs has never been more urgent. Although the number of people receiving antiretroviral drugs in low- and middle-income countries increased 10-fold in the last six years, new infections in 2007 outpaced antiretroviral therapy uptake by a margin of five to two.[4, 5] Randomized controlled trials (RCTs) are generally considered the gold standard to define the evidence base for HIV prevention programs and policies. However, only one in seven RCTs of interventions to prevent sexual transmission of HIV has shown efficacy.[6-12] In fact, the overwhelming majority of completed RCTs are “flat” – unable to demonstrate either a positive or adverse effect.

Flat results may occur for three reasons. The underlying concept may be flawed; the concept may be sound, but the specific intervention approach may be ineffective; or, especially for interventions postulated to have only modest impact, aspects of the study design, implementation, or context may limit detection of a true effect. RCT results due to the first two situations have been critically important to advancing HIV prevention science.[6, 7, 10, 11] The third squanders resources and costs lives. Our review addresses this last phenomenon.

Methods

Search Strategy

We conducted a systematic literature review of phase IIb or III RCTs for prevention of sexual transmission of HIV. We searched PubMed/MEDLINE, EMBASE, the Cochrane Library including the Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science for articles meeting our inclusion criteria as of December 6, 2009. There were no language restrictions to the search. We developed a customized search strategy for each database relying on the database's controlled vocabulary or index (e.g., medical subject headings (MeSH)) or free text terms. In most cases, search strategies combined terms for (1) HIV infection, (2) incidence or hazard, (3) prevention, and (4) study design restrictions (randomized controlled designs). Appropriate MeSH or free text terms were also included to exclude trials that focused on the prevention of mother-to child transmission of HIV. In PubMed/MEDLINE, we searched for clinical trials using Cochrane's “Highly Sensitive Search Strategy” for identifying randomized controlled trials.[13] Search strategies for each database are available from the authors.

We conducted a cited reference search with key articles, scanned reference lists of eligible articles and reviews, and searched the electronic conference proceedings of recent HIV/AIDS-related conferences (Conference on Retroviruses and Opportunistic Infections, International Society for Sexually Transmitted Disease (STD) Research annual meetings, and International AIDS Society annual meetings). Clinicaltrials.gov was searched to identify ongoing or recently completed trials. We contacted study authors and reviewed secondary articles to obtain more information about the trials, as needed. We also searched reference lists of eligible articles and recent commentaries and reviews for potentially relevant trials, and contacted colleagues for additional information not revealed in our search. The broad inclusion criteria and the resulting heterogeneity of interventions precluded the use of meta-analytic methods.

Trial Selection

Eligible trials were those that 1) used randomized controlled designs (individual or community); 2) evaluated interventions focusing on sexual transmission of HIV in non-pregnant populations; and 3) reported HIV incidence as the primary or secondary outcome. Trials that assessed HIV infection as part of an aggregate STD outcome (i.e. any incident STD at follow-up) were excluded.[14-18] We first examined the citations from the literature search to eliminate obviously ineligible studies (e.g., non-randomized studies, studies without biological outcomes, inappropriate article types such as reviews or commentaries). Abstracts were specifically searched for mention of an intervention tested against a control intervention with biological outcomes. Report of any STD outcome in the abstract such as incident gonorrhea or chlamydia infections automatically warranted a full length review of the article to determine if HIV testing was performed. We then conducted a detailed manual review of full length articles to determine eligibility.

Data Extraction

All authors independently reviewed and abstracted data from each eligible article or abstract. Discrepancies among the authors were identified and resolved. In two cases, multiple interventions were described in a single article (i.e., multiple treatment arms with different interventions and a single control).[19, 20] In these instances, each intervention was counted separately as an appropriate comparison could be made to the control. We abstracted descriptive information about each trial and the most adjusted measure of effect on HIV infection and 95% confidence interval (CI), giving preference to intention-to-treat analyses. We computed the measure of effect using standard methods when only the risk or rate was presented stratified by study arm (e.g., incidence in the treatment group divided by the incidence in the comparison group).[21] We also abstracted the measure of effect in pre-planned subgroup analyses among those studies for which we were aware that such analyses had been conducted.

We categorized trials by type of prevention intervention (e.g. behavioral, microfinance, diaphragm, microbicides, pre-exposure prophylaxis (PrEP), male circumcision, sexually transmitted infection (STI) treatment, and vaccines) and their effect on HIV incidence. “Positive” intervention trials significantly reduced the risk of HIV infection in the intervention arm compared to the control arm, whereas “adverse” trials significantly increased HIV risk. Interventions with “no effect” were those in which no statistically significant effect on HIV acquisition or transmission was reported (positive or adverse), thus the null hypothesis could not be rejected. Although we examined effect size and precision of the measures of effect (usually an incidence rate ratio (IRR) or hazard ratio (HR)), we considered statistical significance of the effect measure (α=0.05) as the definitive measure of whether an intervention was positive, adverse, or had no effect.

We also considered a broad set of methodological and contextual factors to identify specific issues in the design, conduct, or environment in which the trials were conducted that might have contributed to their results. We compared the projected and observed HIV incidence in the comparison arm (per 100 person years or annual cumulative incidence) either given by the authors or computed using standard methods.[21] The level of adherence achieved in both arms of the study was also abstracted. In general, adherence in the intervention arms was defined as attendance at intervention activities for behavioral trials, the proportion of sex acts where the study product was used for diaphragm and microbicide trials, the proportion of prescribed doses completed for PrEP and for STI treatment or suppression trials, or the proportion of participants receiving all doses in vaccine trials. Adherence in the control arm was defined similarly for the placebo product or prevention intervention offered to the control group.

In all instances, the intervention arm examined the marginal benefit of the intervention over and above other prevention services in both the intervention and control arm. We assessed whether the these services, often offered to both the intervention and control arms, exceeded the local community standard of care in terms of basic components of HIV prevention such as risk-reduction counseling or HIV education, provision of condoms and related counseling, and availability of STI treatment services or case management. Comparison arms in which all of these components were augmented beyond the local, existing standard of HIV prevention were classified as “exceptional” prevention packages. Comparison arms in which at least one of these components was added or strengthened were classified as “enhanced” prevention, whereas those with nothing above the local standard were classified as “standard of care.” In addition, regardless of the intensity of the prevention services offered, in some cases the type of preventive services provided to the control arm overlapped with the primary intervention being tested, albeit in a diluted form (e.g., for behavioral interventions, improved VCT in the control group compared to enhanced VCT combined with other behavioral interventions in the intervention group). Trials with control groups that overlapped with the main intervention in this way were highlighted as “overlapping”. Finally, to assess the impact of these improved prevention services, we evaluated whether risk behavior decreased in both arms of the study compared to baseline measurements.

Results

Literature Search

The results of the literature search are presented in Figure 1. We identified 3,616 unique citations from electronic databases of which 3,555 were excluded based on title examination and 27 were excluded based on abstract-level review. Thirty-four full-length articles were reviewed in detail. Twenty-nine articles from the literature search met our inclusion criteria. Eight additional studies were identified through reference list examinations, reviews of conference abstracts, or other sources.[7, 22-25] Thus, 37 HIV prevention RCTs were included in this review reporting on 39 unique interventions to prevent sexual transmission of HIV (Table 1). Five studies included in the review were unpublished and obtained from conference presentations or other sources.[20, 25-29]1 Here we summarize characteristics of the prevention trials and then discuss the most salient factors associated with RCTs demonstrating significant results and those with flat results, by type of intervention.

Figure 1
Systematic review flow diagram
Table 1
Descriptive characteristics of 37 randomized controlled trials reporting on 39 interventions for prevention of sexual transmission of HIV infection.

Characteristics of prevention intervention trials

The 37 trials were conducted over approximately 30 years from 1987 to 2009 at a rapidly accelerating pace with more than 80% being initiated since 1995 (Table 1). The majority were implemented partially or entirely in southern Africa (n=32) and employed individually randomized designs (n=26). Six RCTs included sites in the Americas [20, 30-34], three enrolled participants in Asia [35-37], and one trial each involved populations in the Netherlands and Australia.[33, 34] Nine community randomized controlled trials (C-RCTs) evaluated behaioral, microfinance, and STI treatment interventions and constituted more than half of the 16 RCTs in these categories.[19, 27, 38-44] The two remaining RCTs examined intervention impact on HIV transmission in discordant couples.[29, 45]

Most of the RCTs (n=17) were conducted exclusively in women; 13 in female sex workers (FSWs) or women at high risk of HIV infection [31, 35, 36, 46-55] and four in sexually active women in the general population.[20, 25, 56, 57] Sixteen studies evaluated outcomes in both males and females, including three RCTs in adolescents.[26, 27, 38, 40] Together the trials enrolled nearly 160,000 participants.

Trials demonstrating statistically significant effects

Six RCTs, all examining biomedical interventions, delivered definitive results on HIV infection (Table 2).[36, 42, 58-61] Five of the six trials were positive; three RCTs of male circumcision (all of which assessed female-to-male HIV transmission) [58-60], one RCT of enhanced STI management [42], and one vaccine trial.[62] In all three male circumcision trials, a substantial effect size was observed ranging from a 51- 61% reduction in HIV incidence.[58-60] The trial of improved STI management in the Mwanza region of Tanzania demonstrated a 42% reduction in HIV incidence.[42] The prime boost combination of ALVAC-HIV, a recombinant canarypox vectored vaccine, and AIDSVAX B/E, a recombinant gp120 vaccine, lowered the rate of HIV acquisition by 31% among Thai volunteers, although there was no reduction in post-infection HIV plasma viral load.[62] One of the six significant RCTs, a microbicide trial of nonoxynol-9 gel, produced adverse results with a substantial effect size; a 50% increase in HIV incidence.[36]

Table 2
Results of 37 randomized controlled trials reporting on 39 interventions to prevent sexual transmission of HIV.

Statistical power was not a major issue in any of these six trials (Table 3).[36, 42, 58-60] However, despite the large sample size (16,402) in the ALVAC/AIDSVAX vaccine trial [62], the low HIV incidence among this sample of the general population limited study power sufficiently so that both the strict intent to treat and per protocol analyses yielded a non-significant 26% reduction in HIV incidence and even the significant efficacy documented in the modified intent to treat analysis had such a wide confidence interval that questions have been raised about the interpretability of the results. Five of the six studies were powered based on fairly close approximations of the observed HIV incidence in the comparison arm [42, 58-60], and in the nonoxynol-9 gel trial, the observed incidence in the comparison arm was approximately double what had been anticipated (5% annually predicted vs. 10.3 per 100 person-years observed).[36]

Table 3
Factors associated with HIV prevention interventions in randomized controlled trials. Trials with significant effects are shown in bold.

Similarly, poor adherence was not a key factor in most of the significant trials. Male circumcision is a one-time, directly administered procedure for which adherence issues are limited to refraining from intercourse during healing [7, 8, 11] and likewise, adherence was relatively high in the microbicide (70-72%) and vaccine (75-78%) trials.[36, 62] In addition, the male circumcision and vaccine trials share the benefit of having directly observed interventions, substantially eliminating the problem of reporting bias. Although adherence was not reported in the STI treatment trial, it is likely to have been substantial because trial participants were symptomatic patients seeking STI care and health educators visited villages to encourage prompt treatment for symptomatic STIs.

In none of these trials did the control group receive a dilute version of the intervention being tested (i.e. no overlap). Male circumcision, nonoxynol-9 gel and vaccine were not provided in any form to controls, and no additional STI training, drugs, or supervisory or educational visits were provided to the control communities in the Mwanza trial. However, in all of the interventions with the exception of the STI treatment trial [42], the control group received an enhanced or exceptional prevention package beyond the local standard of care.

Trials not demonstrating statistically significant effects

Behavioral interventions

All seven behavioral interventions yielded flat results, however, in the Masaka, Uganda community randomized trial, significant protective effects were found in a subgroup analysis among sexually active women in analyses at the individual-level (IRR=0.41, 95% CI: 0.19, 0.89).[19, 63] Of the four studies that reported both projected and observed HIV incidence in the control arm, all but one observed incidence rates that were substantially lower than original projections. Only in Project EXPLORE did the projected HIV incidence in the control arm accurately reflect that observed in the trial.[30] Thus overall, limited power may have limited the ability to detect an effect.

A central issue common to six of these seven trials is that some form of the intervention overlaps with that provided to the control group, albeit in more dilute form. With one exception (Regai Dzive Shiri) [26, 27] each of these studies offered some combination of risk reduction counseling, condom promotion, and referral and treatment for STIs that exceeded the local standard of care. The effects of so-doing are apparent in the four trials that measured changes in risk behaviors, with all four noting declines in both arms.[19, 30, 40, 64]

The Regai Dzive Shiri youth intervention was a community RCT (see footnote); no prevention services were offered to the comparison communities and uptake of the intervention was low (41%) among survey participants.[27] Adherence was also poor in the Stepping Stones trial, with 44% of participants reporting attendance at 75% or more of the Stepping Stones sessions.[27, 40] Adherence ranged between 70-75% for the other two trials for which adherence data was reported, Project Explore [30] and the Zimbabwean workplace intervention.[39]

Microfinance interventions

To date, there has only been one microfinance RCT that has examined HIV endpoints and there was no effect on HIV incidence in study communities.[41] This may be due, in part, to the indirect nature of the intervention, which was directly offered to women micro-financers whereas the effect on HIV was expected to diffuse to younger women in the populations, which might take years to manifest.[41] The observed HIV incidence was greater than that projected, adherence was moderate (65%), and there was no change in behavior in either study arm.[41]

Microbicide and diaphragm trials

Eleven of the 12 trials of microbicides and the latex diaphragm trial demonstrated flat results. However, of note, 0.5% PRO 2000/5 gel demonstrated a 33% reduction in incidence compared to placebo gel (HR=0.7, 95% CI: 0.5, 1.1, p=0.10) in the intent-to-treat analysis and a 36% reduction compared to no gel (p = 0.04) in the per protocol analysis.[20] In a larger study, PRO 2000 gel had no effect on HIV incidence.[25] Although the overall estimates for the cellulose sulfate [35] and SAVVY (C31G) gel [51] trials were non-significant, subgroup analyses revealed significant adverse results in the interim and per protocol analyses of the cellulose sulfate trial and among women in the SAVVY trial with greater than median coital frequency and greater than median frequency of gel use.

Reduced power to detect an effect resulting from a lower than expected incidence of new infections was apparent in six of the seven flat microbicide trials that reported this information. Adherence ranged between 73-96% among those studies for which adherence was reported (the lower bound reported in the diaphragm trial), with only two achieving rates of at least 85% in the intervention arm.[35, 57] In addition, results from the diaphragm trial indicated differential condom use over the course of the trial with an average of 53% in the intervention arm and 85% in the control.[56] Although none of these studies offered a dilute, overlapping version of the intervention to controls, all of the studies offered exceptional prevention interventions to both arms. With one exception, risk taking behavior decreased in both study arms in all of the trials that reported these data.[51]

Pre-exposure prophylaxis

Only one trial of pre-exposure prophylaxis (PrEP) using ARVs for HIV prevention has been completed.[52] The observed annual HIV incidence was roughly half of what was anticipated, and premature closure of the trial markedly reduced study power. Adherence rates shy of 70% were observed in both arms.[52] Enhanced prevention services were provided to controls and risk behavior was reduced in both study arms throughout the trial.

Male circumcision to prevent male-to-female transmission

A fourth circumcision trial examined HIV transmission to female partners of HIV-infected men who were enrolled in the circumcision RCT in Rakai, Uganda.[45] Overall, no significant reduction in HIV incidence was observed, but HIV acquisition was increased in the subgroup of female partners of men who resumed sexual activity early before complete wound healing compared to those who delayed resumption of sexual activity (RR=2.92, 95% CI: 1.02, 8.46, p=0.06).[45]

STI treatment trials

Eight of the nine trials of STI treatment for HIV prevention delivered flat results, although one study found a significant effect on HIV incidence in a subgroup of males who attended program meetings (adjusted IRR= 0.48, 95% CI: 0.24, 0.98, p= 0.04).[44] Five RCTs evaluated various approaches to improved management of curable STIs, including two of syndromic STI management in the general population (both community RCTs) [19, 44]), two of periodic presumptive therapy (one a CRCT in the general population [43] and the other an individual RCT in female sex workers (FSWs) [54]) and one individual RCT of intensive, microscopy-assisted STI screening in FSWs.[53] The remaining three RCTs tested acyclovir suppressive therapy in both high and low risk populations.[29, 32, 55]

Several factors probably contributed to the striking contrast between the positive results of the Mwanza STI treatment trial and the five flat RCTs that targeted curable STIs. Although low power, poor adherence, overlapping interventions, or other enhanced prevention services in the comparison communities did not appear to be problems in the Mwanza trial, at least one of these issues arose in each of the five flat trials. In addition, observed HIV incidence fell short of that projected in all four of the RCTs that reported this information [19, 43, 44, 54] and adherence was a moderate 81% in the fifth trial.[53] Perhaps more importantly, like the behavioral interventions, all of these trials offered enhanced or exceptional prevention services to controls, including improved STI services (constituting overlap with the primary intervention being tested) in three of the five RCTs.[43, 53, 54] Indeed, these control arm interventions were reflected in decreases in risk behaviors in both arms of all five RCTs. Finally, the Mwanza trial was implemented in an earlier phase HIV epidemic than was the case for the five flat trials of treatment of curable STIs, all of which were conducted in late-phase, generalized epidemics when genital herpes had largely replaced curable etiologies of genital ulcers while rates of other curable STIs had fallen substantially in the general population.

Although epidemic phase was not a concern in the three acyclovir suppression trials, adherence was a challenge, with the proportion of participants reporting taking >90% of pills ranging from 51% to 73% in two of the three RCTs. In addition, exceptional HIV prevention services were available to controls with attendant reductions in risk behaviors in all trial arms.[29, 32, 55] However, growing data suggest that in these trials it is likely that the 400 mg BID acyclovir regimen tested was not capable of extinguishing persistent immune activation or other biological mechanisms triggered by HSV infection, which could increase susceptibility to HIV.[65, 66]

Vaccines

Three vaccine trials yielded flat results [33, 34, 37, 67]; however, subgroup analyses in the Step Study of the Merck Ad5 gag/pol/nef vaccine suggested significant adverse effects among uncircumcised men (HR=3.8, 95% CI 1.5, 9.3) and those with high pre-existing Ad5 antibody titers (HR=2.3, 95% CI 1.2, 4.3).[34]. However, the increased risk of HIV seroconversion associated with preexisting immunity to the Ad5 vector did not persist in an interim analysis of Step Study participants after unblinding, and it was not confirmed in analyses of the Phambili trial of the same candidate in South Africa.[67-69]. Of note, vaccinations in the Phambili trial were stopped prematurely when the results of the Step trial became available and, as a result, the study accrued very limited numbers of HIV seroconverters prior to unblinding, limiting its power to assess this association.

Large sample size and an endpoint-driven design provided adequate power in the one vaccine RCT that was not terminated early.[33] Adherence was also not an issue in these RCTs as vaccine was directly administered and completion of the vaccine series was high (82-90%) in the studies reporting these data. However enhanced or exceptional HIV prevention were offered to controls in all three trials with decreases in risk behaviors in both study arms in the two trials reporting this information.[33, 34]

Discussion

HIV prevention can work, as demonstrated by successes in Thailand and Uganda and now the promise of male circumcision.[58-60, 70, 71] Yet in the face of what continues to be one of the most devastating pandemics we have ever known, new, evidence-based approaches are urgently needed. In this context, the fact that almost 90% of RCTs of interventions for prevention of sexual transmission of HIV have delivered flat results demands careful analysis. This review revealed that the majority of flat RCTs are attributable, at least in part, to issues related to trial design and/or implementation. These issues must be addressed in future intervention research.

Compared to what?

A key issue in evaluating RCTs is the nature of the control group, and in particular, the intensity of prevention services offered to both study arms.[72-75] The introduction of enhanced or exceptional HIV prevention programs in the control arm that exceed the community standard of care (and may even include a diluted version of the primary intervention) are rarely sustainable after trial completion, and their intensity may dramatically reduce the ability to detect the effect of a new and effective intervention. Furthermore, in these cases, the results have little external validity because the comparison does not represent the effect of the intervention compared to the condition actually experienced by individuals in the community who could not avail themselves of the services offered to the control group.

Trials in which investigators or institutional review boards often feel obligated to provide controls with enhancements that are dilute versions of the intervention being tested (such as RCTs of behavioral or STI treatment interventions) are particularly vulnerable to flat results. In contrast, RCTs of new biomedical interventions such as vaccines, microbicides or male circumcision rarely offer dilute forms of the intervention in the comparison group. This makes distinguishing an ineffective intervention from design issues more straightforward. Indeed, in all the trials that had a significant effect (positive or adverse), the control group did not receive a prevention package that resembled the main intervention.

The ethical issues of offering enhanced HIV prevention services in the comparison arm must be weighed against the ethical issues of lengthy and expensive prevention trials that provide the control group with an unsustainable level of prevention services that does not reflect community standards. Further, such trials may jeopardize our ability to identify and offer participants and at-risk individuals around the world additional effective HIV prevention options. Stepped-wedge designs are one approach that, when appropriate, may help address this ethical dilemma.[76] Similarly, a reliable incidence assay might obviate the need for prospectively following controls, in which case the need to offer additional services over the course of the trial might be moot.

We also examined changes in risk behavior over the course of the trial in the intervention and control arms and found that in most cases, risk-taking behavior was reduced in both. Some of this change may be attributable to enhanced prevention services offered in the trial. A “Hawthorne effect”, in which simply participating in the study and being followed produces a positive result, might also have contributed. For example, in a prospective cohort study of female sex workers in Kenya, HIV-1 incidence declined 10-fold during 3 years of follow-up.[77] This phenomenon further attenuates the ability to detect the marginal benefit of the new intervention, especially if it is postulated to have only a modest effect.

Power to detect an effect: incidence and epidemic phase

Much has been written about unexpectedly low incidence in trials [24, 78, 79], which results in low statistical power unless additional endpoints are obtained by extending trial duration and incurring increased costs. We observed that this was the case in 14 (64%) of the 22 flat trials reporting this information. It was also recently announced that the PrEP study in Botswana would be unable to evaluate efficacy given a lower than expected rate of new infections (note this study is in progress and so was not included in this review).[80] Lower than expected incidence could result from erroneous assumptions about incidence prior to beginning the study. Development of improved assays to detect incident HIV infection might improve estimates of incidence before the onset of a trial and reduce trial costs by providing a more accurate basis for sample size calculations and assisting in the more reliable selection of populations with sufficiently high HIV incidence to permit trials of shorter duration.[11]. Alternatively, unexpectedly low incidence could result from changes in risk behaviors during the trial (potentially due to prevention services offered or to a Hawthorne effect). In addition, unexpectedly low incidence could reflect changes in the nature of the local epidemic, driven either by unanticipated implementation of new interventions or by changes in the epidemic phase in which the RCT was conducted – due, in part, to the protracted time for RCT design, implementation and completion, which can take as long as a decade.[20] In such trials, it is important for investigators to document and iteratively reassess their original assumptions.

Effectiveness versus efficacy: does adherence matter

Another key issue is the importance of adherence in trials.[8, 56] If trials assess effectiveness as opposed to efficacy, one might argue that adherence is an inherent part of the intervention and that if it is suboptimal during the trial, it does not bode well for general uptake outside the course of the trial where adherence is likely to be even worse than in the context of a study. Nevertheless, if the methods require high adherence, as is likely to be the case with microbicides, PrEP and treatment regimens for genital herpes, critical priorities are to better define, measure, analyze and routinely report adherence, as well as to identify concomitant behavioral interventions to increase adherence (as we currently do for male condoms) before products are incorporated into HIV prevention packages.

The Way Forward

It is clear from results of recent trials that, in the near-term, there will be no single “magic bullet” for HIV prevention. Instead, the emphasis in prevention research is shifting to evaluation of combination prevention packages in which synergies among interventions with modest levels of effect might lead to substantial efficacy overall.[81, 82] What types of evidence for potential efficacy should be used to select interventions for inclusion in these packages? In addition to approaches such as male circumcision with robust RCT evidence for significant effects, we might consider interventions that demonstrate efficacy among specific, relevant subgroups (e.g., males who attended program meetings in the STI intervention in Zimbabwe [44]). Interventions with significant secondary outcomes in RCTs (e.g. other STDs [3, 83]) might also be candidates for evaluation in combination packages if we assume that these outcomes lie on the causal pathway to HIV infection.[24]

Should we also consider interventions for which evidence from RCTs is mixed or absent, but for which observational data or modeling strongly suggest that both the underlying concept and the specific intervention are likely to deliver substantial protection? Most HIV prevention programs and policies are not currently based on RCT evidence. Public health agencies such as the Centers for Disease Control and Prevention (CDC), the Agency for Healthcare Research and Quality, the World Health Organization and UNAIDS have long recognized the dilemma posed by relying solely on RCT data. Therefore, many develop guidelines (as established, for example, by the U.S. Preventive Services Task Force (USPSTF)) that are based not only on RCTs, but also on observational data from well-designed non-randomized trials; cohort, case-control, or multiple time series studies; mathematical modeling; dramatic results from uncontrolled experiments or even expert opinion by explicitly using a systematic approach to rating the evidence from these different methods.[84] Similarly, epidemiologists and public health practitioners rarely rely only on the evidence from controlled trials to infer causality between an exposure and an outcome. For example, when reviewing evidence from multiple sources to establish whether smoking was associated with lung cancer, Sir Bradford Hill suggested that experimental evidence was only one of several considerations for causal inference. Other criteria included parameters that characterized the nature of the association between the independent and dependent variable including: the magnitude of the effect, consistency, temporality, specificity, and the biological plausibility of the association.[85]

That said, a critical issue is that most RCTs reviewed here are highly managed, labor-intensive and expensive; meeting the standards of regulatory bodies such as the US Food and Drug Administration (FDA). As alluded to above, before abandoning randomization, it is important to consider the entire universe of RCTs including stepped wedge designs, large simple trials, or trials where intervention communities are compared to either what currently exists or where incidence comparisons are made with non-participants as determined by reliable assays (should they be developed). The need for a counterfactual to infer causality cannot be denied and may be especially important when such combination programs are rolled out and evaluated at-scale, particularly if they are designed based on uncertain efficacy.

Randomized controlled trials will undoubtedly remain our gold standard in defining the evidence-base for prevention programs and policies. However, to assess the purity of this gold standard, the HIV prevention science community must not only examine evidence from randomized controlled trials with significant outcomes (including from subgroups and secondary outcomes), but must also examine flat trials and address the design and implementation issues discussed above. In addition, we must acknowledge and explicitly define the role of other types of evidence in the development of HIV prevention recommendations.

Pure gold is a thing of great beauty and value, but lacks the strength and affordability that make alloys like steel so useful and durable. Similarly, well designed and executed RCTs are magnificent and invaluable cornerstones of HIV prevention policies and programs. However, before abandoning entire classes of potentially beneficial interventions, we must forge “alloys” of data from RCTs, observational studies and other lines of evidence; cautiously and explicitly titrating the use of less rigorous sources, and recognize that these “alloys” are likely to offer the best guide to decide what to include in prevention packages, what to scale up, and where further research is warranted.

Acknowledgments

Dr. Wasserheit conceptualized the idea of a “score card” and delivered the initial results in 2007 at the Conference on Retroviruses and Opportunistic Infections reviewing the results of several STI treatment trials to reduce incidence of HIV.[1] Dr. Padian elaborated on this concept and reviewed the larger array of biomedical trials using this paradigm and introduced the importance of “flat trials” in 2007 in a plenary for the International AIDS Society.[2] Together, Drs. Padian and Wasserheit refined the concept and both were involved in all aspects of the review and drafting the manuscript. Ms. Balkus conducted the initial literature review and participated in writing the first draft of the manuscript. Dr. McCoy updated the review, reviewed the abstracted data for accuracy with Ms. Balkus, and participated in writing and revising the manuscript. Dr. Wasserheit presented a preliminary version of this paper in 2009 (co-authored by Drs. Padian and McCoy and Ms. Balkus) at the International Society for STD Research.[3]

Dr. Padian was supported by a grant from the Bill and Melinda Gates Foundation (No. 21082) and Dr. Wasserheit was supported by a grant from the National Institute of Allergy and Infectious Diseases (RO1 AI083034).

Sources of Support: Bill and Melinda Gates Foundation (Dr. Padian, No. 21082)

Footnotes

1Although the Regai Dzive Shiri trial was originally implemented as a community randomized controlled trial with evaluation in a longitudinal cohort, the analysis was re-conceptualized as a serial cross-sectional evaluation due to significant out-migration from trial communities.[26,27] We included this trial in the review, despite not reporting on HIV incidence (HIV prevalence was measured in study communities after the intervention was implemented) because of the size of the study and the important implications this study has for understanding the challenges in scaling up prevention packages.

References

1. Wasserheit JN. The STD-HIV Connection from Research to Action: Are We Lost in Translation?; 14th Conference on Retroviruses and Opportunistic Infections; Los Angeles, California. 2007.
2. Padian NS. Biomedical HIV Prevention: Current Status, Challenges, and Future Directions. 4th Annual International AIDS Society; Syndney: 2007.
3. Wasserheit JN. STI/HIV Prevention Research: The Agony, The Ecstasy, & The Future. 18th International Society for Sexually Transmitted Disease Research; London: 2009.
4. Lay PD. AIDS remains an exceptional issue. BMJ Rapid Response. 2008
5. UNAIDS. Report on the Global AIDS Epidemic. Geneva: 2008.
6. Institute of Medicine (IOM) Methodological Challenges in Biomedical HIV Prevention Trials. Washington, D.C.: The National Academies Press; 2008.
7. Padian NS, Buve A, Balkus J, Serwadda D, Cates W., Jr Biomedical interventions to prevent HIV infection: evidence, challenges, and way forward. Lancet. 2008;372:585–599. [PubMed]
8. Weiss HA, Wasserheit JN, Barnabas RV, Hayes RJ, Abu-Raddad LJ. Persisting with prevention: The importance of adherence for HIV prevention. Emerg Themes Epidemiol. 2008;5:8. [PMC free article] [PubMed]
9. Potts M, Halperin DT, Kirby D, Swidler A, Marseille E, Klausner JD, et al. Public health. Reassessing HIV prevention. Science. 2008;320:749–750. [PMC free article] [PubMed]
10. Grant RM, Hamer D, Hope T, Johnston R, Lange J, Lederman MM, et al. Whither or wither microbicides? Science. 2008;321:532–534. [PMC free article] [PubMed]
11. Lagakos SW, Gable AR. Challenges to HIV prevention--seeking effective measures in the absence of a vaccine. N Engl J Med. 2008;358:1543–1545. [PubMed]
12. Grosskurth H, Gray R, Hayes R, Mabey D, Wawer M. Control of sexually transmitted diseases for HIV-1 prevention: understanding the implications of the Mwanza and Rakai trials. Lancet. 2000;355:1981–1987. [PubMed]
13. Higgins JPT, Green S, editors. The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [updated September 2008] 2008.
14. Kamb ML, Fishbein M, Douglas JM, Jr, Rhodes F, Rogers J, Bolan G, et al. Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases: a randomized controlled trial. Project RESPECT Study Group. JAMA. 1998;280:1161–1167. [PubMed]
15. Hobfoll SE, Jackson AP, Lavin J, Johnson RJ, Schroder KE. Effects and generalizability of communally oriented HIV-AIDS prevention versus general health promotion groups for single, inner-city women in urban clinics. J Consult Clin Psychol. 2002;70:950–960. [PubMed]
16. Branson BM, Peterman TA, Cannon RO, Ransom R, Zaidi AA. Group counseling to prevent sexually transmitted disease and HIV: a randomized controlled trial. Sex Transm Dis. 1998;25:553–560. [PubMed]
17. Baker SA, Beadnell B, Stoner S, Morrison DM, Gordon J, Collier C, et al. Skills training versus health education to prevent STDs/HIV in heterosexual women: a randomized controlled trial utilizing biological outcomes. AIDS Educ Prev. 2003;15:1–14. [PubMed]
18. Pequagnat W. Group NCHSPT. Results of the RCT to test the community popular opinion leader (C-POL) intervention in five countries. AIDS 2008 - XVII international AIDS Conference: Abstract no LBPE1166
19. Kamali A, Quigley M, Nakiyingi J, Kinsman J, Kengeya-Kayondo J, Gopal R, et al. Syndromic management of sexually-transmitted infections and behaviour change interventions on transmission of HIV-1 in rural Uganda: a community randomised trial. Lancet. 2003;361:645–652. [PubMed]
20. Karim SA, Coletti A, Richardson B, Ramjee G, Hoffman I, Chirenje M, et al. Safety and Effectiveness of Vaginal Microbicides BufferGel and 0.5% PRO 2000/5 Gel for the Prevention of HIV Infection in Women: Results of the HPTN 035 Trial; 16th Conference on Retroviruses and Opportunistic Infections; Montreal, Canada. 2009.
21. Rothman KJ, Greenland S. Modern Epidemiology. 2nd. Philadelphia: Lippincott Williams & Wilkins; 1998.
22. Cohen MS. Preventing sexual transmission of HIV. Clin Infect Dis. 2007;45(4):S287–292. [PubMed]
23. Vermund SH, Allen KL, Karim QA. HIV-prevention science at a crossroads: advances in reducing sexual risk. Curr Opin HIV AIDS. 2009;4:266–273. [PMC free article] [PubMed]
24. Gray RH. Methodologies for evaluating HIV prevention intervention (populations and epidemiologic settings) Curr Opin HIV AIDS. 2009;4:274–278. [PMC free article] [PubMed]
25. Medical Research Council. Microbicide Gel Safe but Not Effective in Preventing HIV Infection. Press Release; Durban: Dec 14, 2009.
26. Cowan FM, Pascoe SJ, Langhaug LF, Dirawo J, Chidiya S, Jaffar S, et al. The Regai Dzive Shiri Project: a cluster randomised controlled trial to determine the effectiveness of a multi-component community-based HIV prevention intervention for rural youth in Zimbabwe--study design and baseline results. Trop Med Int Health. 2008;13:1235–1244. [PubMed]
27. Cowan FM, Pascoe SJS, Langhaug LF, Dirawo J, Mavhu W, Chidiya S, et al. 18th International Society for Sexually Transmitted Disease Research. London: 2009. The Regai Dzive Shiri Project: Results of a Cluster Randomized Trial of a Multi-Component HIV Prevention Intervention for Rural Zimbabwean Adolescents.
28. Gray GE, Allen MA, Bekker L, Churchyard G, Mlisana K, Nchabeleng M, et al. AIDS Vaccine. Cape Town: 2008. Results from the Phambili (HVTN 503) Study: A Multicenter Phase IIB Test-of-concept Study of the MRKad5 HIV-1 Gag/Pol/Nef Vaccine in South Africa.
29. Celum C, Wald A, Lingappa J, Magaret A, Wang R, Mugu N, et al. Twice-daily acyclovir to reduce HIV-1 transmission from HIV-1/HSV-2 co-infected persons within HIV-1 serodiscordant couples: a randomized, double-blind, placebo-controlled trial; 5th IAS Conference on HIV Pathogenesis, Treatment, and Prevention; Cape Town, South Africa. 2009.
30. Koblin B, Chesney M, Coates T. Effects of a behavioural intervention to reduce acquisition of HIV infection among men who have sex with men: the EXPLORE randomised controlled study. Lancet. 2004;364:41–50. [PubMed]
31. Patterson TL, Orozovich P, Semple SJ, Orozco HSS, Fraga M, Amaro H, et al. A Sexual Risk Reduction Intervention for Female Sex Workers in Mexico: Design and Baseline Characteristics. Journal of HIV/AIDS & Social Services. 2006;5:115–137.
32. Celum C, Wald A, Hughes J, Sanchez J, Reid S, Delany-Moretlwe S, et al. Effect of aciclovir on HIV-1 acquisition in herpes simplex virus 2 seropositive women and men who have sex with men: a randomised, double-blind, placebo-controlled trial. Lancet. 2008;371:2109–2119. [PMC free article] [PubMed]
33. Flynn NM, Forthal DN, Harro CD, Judson FN, Mayer KH, Para MF. Placebo-controlled phase 3 trial of a recombinant glycoprotein 120 vaccine to prevent HIV-1 infection. J Infect Dis. 2005;191:654–665. [PubMed]
34. Buchbinder SP, Mehrotra DV, Duerr A, Fitzgerald DW, Mogg R, Li D, et al. Efficacy assessment of a cell-mediated immunity HIV-1 vaccine (the Step Study): a double-blind, randomised, placebo-controlled, test-of-concept trial. Lancet. 2008;372:1881–1893. [PMC free article] [PubMed]
35. Van Damme L, Govinden R, Mirembe FM, Guedou F, Solomon S, Becker ML, et al. Lack of effectiveness of cellulose sulfate gel for the prevention of vaginal HIV transmission. N Engl J Med. 2008;359:463–472. [PubMed]
36. Van Damme L, Ramjee G, Alary M, Vuylsteke B, Chandeying V, Rees H, et al. Effectiveness of COL-1492, a nonoxynol-9 vaginal gel, on HIV-1 transmission in female sex workers: a randomised controlled trial. Lancet. 2002;360:971–977. [PubMed]
37. Rerks-Ngarm S, Pitisuttithum P, Nitayaphan S, Kaewkungwal J, Chiu J, Paris R, et al. Vaccination with ALVAC and AIDSVAX to Prevent HIV-1 Infection in Thailand. N Engl J Med. 2009 [PubMed]
38. Ross DA, Changalucha J, Obasi AI, Todd J, Plummer ML, Cleophas-Mazige B, et al. Biological and behavioural impact of an adolescent sexual health intervention in Tanzania: a community-randomized trial. AIDS. 2007;21:1943–1955. [PubMed]
39. Corbett EL, Makamure B, Cheung YB, Dauya E, Matambo R, Bandason T, et al. HIV incidence during a cluster-randomized trial of two strategies providing voluntary counselling and testing at the workplace, Zimbabwe. AIDS. 2007;21:483–489. [PubMed]
40. Jewkes R, Nduna M, Levin J, Jama N, Dunkle K, Puren A, Duvvury N. Impact of stepping stones on incidence of HIV and HSV-2 and sexual behaviour in rural South Africa: cluster randomised controlled trial. BMJ. 2008;337:a506. [PMC free article] [PubMed]
41. Pronyk PM, Hargreaves JR, Kim JC, Morison LA, Phetla G, Watts C, et al. Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: a cluster randomised trial. Lancet. 2006;368:1973–1983. [PubMed]
42. Grosskurth H, Mosha F, Todd J, Mwijarubi E, Klokke A, Senkoro K, et al. Impact of improved treatment of sexually transmitted diseases on HIV infection in rural Tanzania: randomised controlled trial. Lancet. 1995;346:530–536. [PubMed]
43. Wawer MJ, Sewankambo NK, Serwadda D, Quinn TC, Paxton LA, Kiwanuka N, et al. Control of sexually transmitted diseases for AIDS prevention in Uganda: a randomised community trial. Rakai Project Study Group. Lancet. 1999;353:525–535. [PubMed]
44. Gregson S, Adamson S, Papaya S, Mundondo J, Nyamukapa CA, Mason PR, et al. Impact and process evaluation of integrated community and clinic-based HIV-1 control: a cluster-randomised trial in eastern Zimbabwe. PLoS Med. 2007;4:e102. [PMC free article] [PubMed]
45. Wawer MJ, Makumbi F, Kigozi G, Serwadda D, Watya S, Nalugoda F, et al. Circumcision in HIV-infected men and its effect on HIV transmission to female partners in Rakai, Uganda: a randomised controlled trial. Lancet. 2009;374:229–237. [PMC free article] [PubMed]
46. Kreiss J, Ngugi E, Holmes K, Ndinya-Achola J, Waiyaki P, Roberts PL, et al. Efficacy of nonoxynol 9 contraceptive sponge use in preventing heterosexual acquisition of HIV in Nairobi prostitutes. JAMA. 1992;268:477–482. [PubMed]
47. Roddy RE, Zekeng L, Ryan KA, Tamoufe U, Weir SS, Wong EL. A controlled trial of nonoxynol 9 film to reduce male-to-female transmission of sexually transmitted diseases. N Engl J Med. 1998;339:504–510. [PubMed]
48. Richardson BA, Lavreys L, Martin HL, Jr, Stevens CE, Ngugi E, Mandaliya K, et al. Evaluation of a low-dose nonoxynol-9 gel for the prevention of sexually transmitted diseases: a randomized clinical trial. Sex Transm Dis. 2001;28:394–400. [PubMed]
49. Peterson L, Nanda K, Opoku BK, Ampofo WK, Owusu-Amoako M, Boakye AY, et al. SAVVY (C31G) gel for prevention of HIV infection in women: a Phase 3, double-blind, randomized, placebo-controlled trial in Ghana. PLoS ONE. 2007;2:e1312. [PMC free article] [PubMed]
50. Halpern V, Ogunsola F, Obunge O, Wang CH, Onyejepu N, Oduyebo O, et al. Effectiveness of cellulose sulfate vaginal gel for the prevention of HIV infection: results of a Phase III trial in Nigeria. PLoS ONE. 2008;3:e3784. [PMC free article] [PubMed]
51. Feldblum PJ, Adeiga A, Bakare R, Wevill S, Lendvay A, Obadaki F, et al. SAVVY vaginal gel (C31G) for prevention of HIV infection: a randomized controlled trial in Nigeria. PLoS ONE. 2008;3:e1474. [PMC free article] [PubMed]
52. Peterson L, Taylor D, Roddy R, Belai G, Phillips P, Nanda K, et al. Tenofovir disoproxil fumarate for prevention of HIV infection in women: a phase 2, double-blind, randomized, placebo-controlled trial. PLoS Clin Trials. 2007;2:e27. [PMC free article] [PubMed]
53. Ghys PD, Diallo MO, Ettiegne-Traore V, Satten GA, Anoma CK, Maurice C, et al. Effect of interventions to control sexually transmitted disease on the incidence of HIV infection in female sex workers. AIDS. 2001;15:1421–1431. [PubMed]
54. Kaul R, Kimani J, Nagelkerke NJ, Fonck K, Ngugi EN, Keli F, et al. Monthly antibiotic chemoprophylaxis and incidence of sexually transmitted infections and HIV-1 infection in Kenyan sex workers: a randomized controlled trial. JAMA. 2004;291:2555–2562. [PubMed]
55. Watson-Jones D, Weiss HA, Rusizoka M, Changalucha J, Baisley K, Mugeye K, et al. Effect of herpes simplex suppression on incidence of HIV among women in Tanzania. N Engl J Med. 2008;358:1560–1571. [PMC free article] [PubMed]
56. Padian NS, van der Straten A, Ramjee G, Chipato T, de Bruyn G, Blanchard K, et al. Diaphragm and lubricant gel for prevention of HIV acquisition in southern African women: a randomised controlled trial. Lancet. 2007;370:251–261. [PMC free article] [PubMed]
57. Skoler-Karpoff S, Ramjee G, Ahmed K, Altini L, Plagianos MG, Friedland B, et al. Efficacy of Carraguard for prevention of HIV infection in women in South Africa: a randomised, double-blind, placebo-controlled trial. Lancet. 2008;372:1977–1987. [PubMed]
58. Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, Nalugoda F, et al. Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial. Lancet. 2007;369:657–666. [PubMed]
59. Bailey RC, Moses S, Parker CB, Agot K, Maclean I, Krieger JN, et al. Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial. Lancet. 2007;369:643–656. [PubMed]
60. Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, Puren A. Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: the ANRS 1265 Trial. PLoS Med. 2005;2:e298. [PMC free article] [PubMed]
61. U.S. Military HIV Research Program. RV144 Phase III HIV Vaccine Trial Press Release. Rockville, MD: 2009.
62. Rerks-Ngarm S, Brown AE, Khamboonruang C, Thongcharoen P, Kunasol P. HIV/AIDS preventive vaccine ‘prime-boost’ phase III trial: foundations and initial lessons learned from Thailand. AIDS. 2006;20:1471–1479. [PubMed]
63. Quigley MA, Kamali A, Kinsman J, Kamulegeya I, Nakiyingi Miiro J, Kiwuwa S, et al. The impact of attending a behavioural intervention on HIV incidence in Masaka, Uganda. AIDS (London, England) 2004;18:2055–2063. [PubMed]
64. Patterson TL, Mausbach B, Lozada R, Staines-Orozco H, Semple SJ, Fraga-Vallejo M, et al. Efficacy of a brief behavioral intervention to promote condom use among female sex workers in Tijuana and Ciudad Juarez, Mexico. Am J Public Health. 2008;98:2051–2057. [PMC free article] [PubMed]
65. Delany S, Mlaba N, Clayton T, Akpomiemie G, Capovilla A, Legoff J, et al. Impact of aciclovir on genital and plasma HIV-1 RNA in HSV-2/HIV-1 co-infected women: a randomized placebo-controlled trial in South Africa. AIDS. 2009;23:461–469. [PMC free article] [PubMed]
66. Zhu J, Hladik F, Woodward A, Klock A, Peng T, Johnston C, et al. Persistence of HIV-1 receptor-positive cells after HSV-2 reactivation is a potential mechanism for increased HIV-1 acquisition. Nat Med. 2009 [PMC free article] [PubMed]
67. Gray G, Allen M, Churchyard G, Bekker L, Nchabeleng M, Mlisana K, et al. AIDS Vaccine. Paris: 2009. A multicenter double-blind randomized placebo-controlled Phase IIB test-of-concept study to evaluate the safety and efficacy of a 3-dose regimen of the Clade B-based Merck Adenovirus serotype 5 HIV-1 gag/pol/nef vaccine in HIV-1-uninfected adults in South Africa.
68. Corey L, McElrath MJ, Kublin JG. Post-step modifications for research on HIV vaccines. AIDS. 2009;23:3–8. [PMC free article] [PubMed]
69. Buchbinder S. AIDS Vaccine. Paris: 2009. Clinical Outcomes from the STEP Study.
70. UNAIDS. UNAIDS Case Study. Geneva: 2000. Evaluation of the 100% Condom Programme in Thailand.
71. Green EC, Halperin DT, Nantulya V, Hogle JA. Uganda's HIV prevention success: the role of sexual behavior change and the national response. AIDS Behav. 2006;10:335–346. discussion 347-350. [PMC free article] [PubMed]
72. Potts M. Thinking about vaginal microbicide testing. Am J Public Health. 2000;90:188–190. [PubMed]
73. van de Wijgert J, Elias C, Ellertson C, McGrory E, Blanchard K, Friedland B, et al. Condom promotion in microbicide trials. Am J Public Health. 2000;90:1153–1154. author reply 1156. [PubMed]
74. Lurie P, Wolfe SM. Ethics require the inclusion of condoms and counseling in anti-HIV microbicide trials. Am J Public Health. 2000;90:1154–1155. author reply 1156. [PubMed]
75. de Zoysa I, Elias CJ, Bentley ME. Microbicide research and “the investigator's dilemma” Am J Public Health. 2000;90:1155. author reply 1156. [PubMed]
76. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54. [PMC free article] [PubMed]
77. Baeten JM, Richardson BA, Martin HL, Jr, Nyange PM, Lavreys L, Ngugi EN, et al. Trends in HIV-1 incidence in a cohort of prostitutes in Kenya: implications for HIV-1 vaccine efficacy trials. J Acquir Immune Defic Syndr. 2000;24:458–464. [PubMed]
78. Halpern V, Obunge O, Ogunsola F, Otusanya S, Umo-Otong J, Wang CH, Mehta N. Interim data monitoring to enroll higher-risk participants in HIV prevention trials. BMC Med Res Methodol. 2009;9:44. [PMC free article] [PubMed]
79. Kaul R, Kimani J, Nagelkerke NJ, Fonck K, Keli F, MacDonald KS, et al. Reduced HIV risk-taking and low HIV incidence after enrollment and risk-reduction counseling in a sexually transmitted disease prevention trial in Nairobi, Kenya. J Acquir Immune Defic Syndr. 2002;30:69–72. [PubMed]
80. Centers for Disease Control and Prevention. CDC Fact Sheet. Atlanta, Georgia: Dec, 2009. Status of the Botswana TDF2 Study of Pre-Exposure Prophylaxis for HIV Prevention.
81. Merson M, Padian N, Coates TJ, Gupta GR, Bertozzi SM, Piot P, et al. Combination HIV prevention. Lancet. 2008;372:1805–1806. [PubMed]
82. Piot P, Bartos M, Larson H, Zewdie D, Mane P. Coming to terms with complexity: a call to action for HIV prevention. Lancet. 2008;372:845–859. [PubMed]
83. Manhart LE, Holmes KK. Randomized controlled trials of individual-level, population-level, and multilevel interventions for preventing sexually transmitted infections: what has worked? J Infect Dis. 2005;191(1):S7–24. [PubMed]
84. U.S. Preventive Services Task Force. Guide to Clinical Preventive Services. 2nd. Baltimore: Williams & Wilkins; 1996. [PubMed]
85. Hill AB. The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine. 1965;58:295–300. [PMC free article] [PubMed]
86. Long-term Evaluation of the MEMA kwa Vijana Adolescent Sexual Health Programme in Rural Mwanza, Tanzania: a Randomised Controlled Trial. Technical Briefing Paper: No 7. 2008
87. Hargreaves JR, Bonell CP, Morison LA, Kim JC, Phetla G, Porter JD, et al. Explaining continued high HIV prevalence in South Africa: socioeconomic factors, HIV incidence and sexual behaviour change among a rural cohort, 2001-2004. AIDS. 2007;21(7):S39–48. [PubMed]