Effectiveness trials for vaginal gel microbicides have several sources of efficacy dilution, most or all of which are present in trials that have been completed so far. We have illustrated that when all these sources of dilution are present within the same trial, a substantial dilution of the true microbicide efficacy may ensue. Under the levels of dilution reported from completed microbicide trials the overall expected effectiveness will be in the range of 24% and 33% when the true efficacy of active gel is 50%. Under more severe dilution conditions, the expected effectiveness could be as low as 16%.
Our analysis has several limitations. Our model is simple and relies on an assumption of weak association between the different dilution factors. Although more complex models could be developed, it would require several assumptions on the relationships between the different sources of dilution. Those additional assumptions could be justified on the basis of the associations observed in microbicide trials. Nevertheless, our findings on the dilution effect due to adherence alone replicate those by Weiss et al [7
] obtained using a different model. For 80% adherence and true efficacies of 25%, 50%, and 75%, they have obtained expected effectiveness of 18%, 37%, and 57%, respectively, compared to 20%, 40%, and 60% under our model. Therefore, the use of the Weiss et al model for the dilution due to low adherence would lead to slightly larger dilution effect. Our definition of adherence includes women who are using the gel inside the appropriate time interval before intercourse as prescribed by the protocol (e.g., within two hours prior to intercourse) and assumes that the efficacy of a product used outside the time interval would be greatly diminished or even completely eliminated. To our knowledge, no reliable data is available on gel applications that are outside the time interval, but further dilution can occur if gel users are not using the product within the correct time interval. As well, we have not considered other potential dilution factors such as product sharing among trial participants who are assigned to different arms of the trial. Only anecdotal evidence has been reported so far, and data on product sharing remains scarce or even inexistent. In addition, further dilution of efficacy can result from a strong association between condom use and microbicide use as described in Trussell and Dominik [14
]. Given the high efficacy of condoms, most of the HIV infections that are observed in a trial are from vaginal (or anal) intercourse unprotected by condoms. Therefore, a positive association between condom and gel use would lead to stronger dilution effects than those presented in this manuscript. Similarly, other behavioral factors which affect adherence might lead to stronger/weaker dilution effects. For instance, if gel users tend to be not very sexually active then stronger dilution effect might be expected since most of the HIV infections in the trial will happen among sexually active women.
Importantly, dilution effects could also reduce the effect of a (truly) harmful product which increases the risk of HIV infections. Thus, some of the candidate microbicides which have been show to increase the risk of HIV acquisition may have been more harmful than what was observed in these trials.
A case in point is the Carraguard trial which specifically discussed and attributed the lack of effectiveness to low adherence, HIV infections from AI, and the potential efficacy of the placebo gel used in the trial [2
]. Based on a staining assay technique, they have estimated the adherence in the trial to be only 42% (although this estimate might be an under-estimate [15
]). Assuming a true efficacy of 50% in our model, an overall expected effectiveness of only 14.9% is obtained with an adherence of 40%, 20% of HIV infections from AI, a placebo efficacy of 10%, and 0% time off-product (pregnant women were discontinued permanently from this trial). The observed effectiveness in the trial was 13% (95% CI [-9%:31%]). Given the potential amount of efficacy dilution in the trial, there was very little chance of detecting the effectiveness of the product if the true efficacy of Carraguard was in the range of 40%-60%. Furthermore, if the true efficacy was in the range of 75%, the overall expected effectiveness under our model would be only 25%, which remains much lower than the 33% the trial was powered to detect. Therefore, if Carraguard was truly moderately efficacious it is highly unlikely that this trial could have detected it. As a consequence of its observed effectiveness, Carraguard may well be discarded from the development pipeline. However, it is interesting to speculate whether it would meet the same fate had it been evaluated under more controlled sources of dilution.
Even though a moderately effective microbicide (30% to 50%) may not be sufficient to eradicate the HIV epidemic, it is nevertheless important to measure the extent of dilution to identify potentially efficacious products for further development, particularly as several moderately effective products with different mechanisms of action could be combined into a single very effective product (e.g., combining a barrier with an entry inhibitor).
Currently planned trials for new candidate microbicides have not fully resolved the problems related to dilution. To reduce the impact of time off-product due to pregnancy, some trials are now requiring that women use two methods of contraception before being allowed to enroll. Given the lack of safety data, it is not possible to allow women to remain on-product while pregnant. However, safety microbicide studies on pregnant women are currently on-going and more are being planned but it will take several years before the results become available from these studies.
Optimizing product adherence is essential. One key issue is the lack of a reliable assessment tool. So far, most of the assessment tools used in microbicide trials have relied on self-reported product use obtained via face-to-face interviews, which has been shown to produce over-estimation of adherence. Newly designed microbicide trials are planning to assess adherence using better assessment tools (e.g., audio computer-assisted self-interview) in order to increase the validity of the adherence data, which may also help to develop strategies to maximize and optimize adherence.
As with adherence, greater effort should be placed to use validated tools to collect data on AI. The dilution effect due to HIV infections that are not from vaginal intercourse is potentially large. Trial data cannot distinguish between anal and vaginal transmission. It is therefore important to improve estimates of the frequency of unprotected AI and to greatly intensify the counseling message to reduce the practice of unprotected AI during the trial. Ultimately, the best option would be to have a candidate product that can be used vaginally and anally as it would potentially provide the greatest benefit. Development of microbicides for AI is currently underway although most products are for vaginal use only.
A suggestion to control for all these different sources of dilutions is that the ITT analysis should not be the primary analysis in a microbicide trial and that so called 'per-protocol' analysis should be used where non-adherent women and time off-product are excluded from the analysis. We think that this is not a viable solution as it is problematic for two reasons. First, the benefit induced by randomization under an ITT analysis often outweighs what can be gained from a non-ITT analysis. For this reason, regulators (such as FDA) typically put a lot of weight on the ITT analysis. It is difficult to foresee how a candidate microbicide could be marketed solely based on its favorable results on a 'per-protocol' analysis with non-significant ITT results. Second, exclusion from the analysis of non-adherent women and/or of women engaging in unprotected AI needs to be done with a validated tool for assessing adherence and risk behaviors which has proven to be a great challenge so far.
Finally, a careful selection of the effect size to be detected in light of the potential dilution factors should be made in the planning of future trials. Protocol teams have used different effect size as it can be seen in past microbicide trial designs where efficacy varying from 30% to 50% have been used to determine the sample size.