Data source: *Encontros*

From 2003 to 2005, *Encontros*, or “coming together”, aimed to decrease incident STI and encourage adoption of consistent condom use among female, male, and transvestite sex workers in Corumbá, Brazil, by using joint clinical and social intervention strategies. The project was designed to engage the sex workers on an individual level through participation in STI/HIV counseling and testing, on an interpersonal level through peer-education, and on a community level through outreach and social activities. Community-based activities were designed to extend and strengthen collegial relationships through providing sex workers opportunities to engage in dialogue around sex work, discrimination, human rights, and prevention.^{24}

Participants included 420 men, women, and transvestites, 18 years of age or more, self-identifying as sex workers, who spoke Portuguese or Spanish and did not plan on leaving the study area permanently in the month following recruitment. Participation included an enrollment visit and four scheduled follow-up visits at 3, 6, 9 and 12 months following enrollment. Each visit included administration of a structured, interviewer-administered questionnaire, counseling for STI/HIV, collection of biological samples for STI testing, treatment for STI if indicated, and testing for HIV if requested. Intervention activities were ongoing during rolling enrollment, follow-up visits, and through the final month of participation. Chlamydia and gonorrhea were diagnosed using COBAS AMPLICOR™ CT/NG PCR (Roche Molecular Diagnostics, Pleasanton, CA).

All enrolled participants were encouraged to participate in project sponsored events; as a result there is no “control” population. Instead we compare participants who were more and less exposed to the intervention, with the hypothesis that sex workers who actively participated in the project would present with fewer incident cases of chlamydia and gonorrhea. Project participation varied at each time point and included contact with peer educators and counselors and participation in community cultural or social events, workshops, educational activities, and organizations/associations. Participation was summarized into a dichotomous indicator of low or high participation at each visit.^{24} Low exposure describes those who attended scheduled appointments but who participated very little in project activities. The high exposure group attended scheduled appointments and had more contact with educators or counselors and participated in project events, workshops, or organizations.

Biases and confounding

Participants in social or community interventions such as *Encontros* often can not be randomized. Instead, they choose how much to participate (or self-select) based on factors often associated with access to services, behaviors, and infection. For example, people who chose to participate more in *Encontros* activities tended to be older and already participating in community groups. Comparison of people participating more and less is likely to be confounded. Similarly, participants who remain in a prevention program or a research cohort are different from those who do not. Over half of the 420 participants in the *Encontros* cohort were lost-to-follow-up prior to completing a year of follow-up. Those who remained in the study were more likely to be married, were less mobile, and reported more STI symptoms. This loss-to-follow-up can be thought of as a missing data problem (absence of data on those who left) or as a selection bias problem (people who stay are different from those who do not). Estimates of effect based only on people who remain in the cohort may not reflect the true program effect on all participants.

Furthermore, in longitudinal studies one may also find time dependent confounding, whereby past outcomes, covariates and exposure to the intervention may impact subsequent exposure and subsequent infections.^{13, 14, 17} Attempting to eliminate the effects of time dependent confounding by treating prior participation as a confounder in a regression model can result in bias by controlling away the effect of interest. However, failure to account for prior participation can also bias the effect estimate. In this study, we explored confounding within intervals of data as well as time dependent confounding. The weighting approach presented herein can mitigate self-selection, loss-to-follow-up and confounding, including time dependent confounding, when relevant.

Weighting to balance populations

Our aim is to assess the effect of the intervention on incident STI among sex workers. We estimate the intervention effect by comparing the odds of presenting with an incident STI if all sex workers in our study population were a high participator, as compared to the odds of an STI if everyone were a low participator at each time point. This can be described as the marginal probability of the outcome under the different levels of participation (often generically called “treatments” in the causal inference literature). We assume that the characteristics that predict participation in the intervention are the same over all time points, or intervals, so we simply pool data over time, analyzing all of the time points together in one model.

Hypothetically speaking, to do away with the biases in this longitudinal data set, one would need to randomize people to low or high participation levels at every visit to ensure that past experiences and covariate distribution had no bearing on the exposure/outcome relationship at each interval. We simulate this repeat randomization using weights. Inverse probability weighting for exposure (or in this case for participation) assigns a weight for each subject equivalent to the inverse probability of being in his/her participation group, based on values of covariates, past outcomes, and exposures at each interval. This treatment probability is estimated using regression techniques. The exposure weights are then applied to the observed population, creating a new pseudo-population in which the joint distribution of confounders is balanced between the two exposure groups, essentially mimicking a randomization procedure at each time point.^{17}

This weighting technique can also be used to deal with the issue of loss-to-follow-up or censoring. We assign each participant a weight equivalent to the inverse probability of remaining in the study at each interval, based on values of observed covariates and past outcomes and exposures. Probabilities are estimated using basic regression techniques. The censoring weights are applied to the observed population, creating a new pseudo-population in which censored subjects are “replaced” by up-weighting uncensored subjects with the same values of past exposures and covariates. Censoring weights and treatment weights are combined (multiplied) into final weights to allow for simultaneous adjustment.

Weighting Procedures

Data management Cohort participants come for repeated visits; the data need to be in long or stacked format () and outcomes must be linked with time points to create treatment weights and run analyses. In the *Encontros* study, not all subjects adhered to their scheduled visit dates, resulting in non-uniformity in data collection. Because visits were to take place every three months, we divided data into three month intervals, centered on the scheduled time points of 3, 6, 9, and 12 month visits, such that each participant's first visit was equivalent to time “0” and subsequent visits between 45 days and 135 days later (3 month intervals of 90 days centered on the next scheduled visit) were included in interval 1, and so on. We include five intervals of data in this analysis.

displays a spreadsheet view of how the data should be set up for a weighted longitudinal analysis. A row has been created for each participant for each interval. Every row indicates whether the participant was present (used to create censoring weights) and, if so, his/her participation level and outcome. Note that there are no exposure data for the first (enrollment) visit; data for analyzing the exposure/outcome relationship begin in interval 1.

Variable and model selection Once the data have been arranged, the next step is to model the association between confounders and the *exposure* (i.e. intervention participation.) When there is loss-to-follow-up or censoring, the association between confounders and presence at each visit must also be modeled. These models are the first step in calculating the weights and are generated using so-called black box techniques designed to do a good job predicting intervention exposure without concern about interpretability, at once attempting to trade-off variance and bias in the model predictions. Just as in a typical regression analysis, variables selected to control for confounding in these treatment models should be potential confounders related to the outcome. Including variables that are strongly related to treatment but not the outcome can severely hurt estimation. The resulting model has two elements: 1) the set of variables chosen, and 2) the functional form of these variables (e.g., main effects, polynomial terms, multiplicative interactions). These models can be chosen by automated algorithms or by expert knowledge, although expert knowledge can rarely identify the proper functional form of the variables.

Most investigators confront missing data. We chose to carry forward values for those covariates deemed not to change drastically in the period of six months. For example, we considered that self-efficacy for condom use would not change at every visit and therefore need not be measured at every visit. To include the self-efficacy variable in the treatment and censoring models, values were carried forward from the last visit. Other methods of data imputation are available.^{25} Variables listed in were considered for inclusion in the weighting models.

| **Table 2**Variable selection for weighting models |

Once a group of variables has been identified for weighting models, various approaches can be taken to model specification and selection.^{26, 27} For these data, we constructed four sets of weights for treatment and censoring using different criteria for each. () First, we chose covariates based on expert knowledge of the subject matter. The second approach was based on an automated algorithm in the program R^{28} called the deletion substitution algorithm (DSA). DSA is a data-adaptive estimation procedure based on cross-validation, which chooses among models trying to minimize the mean-squared error of prediction.^{29, 30} The third approach utilized another automated algorithm in R called polyclass, which builds a hierarchical set of models with main effects, linear spline terms and interaction terms and uses cross-validation to determine the optimal model size.^{31} We also constructed weights using a stepwise selection process, including all first order variables (main effects) and fractional polynomial terms (for continuous variables) with a p-value < 0.1.

| **TABLE 3**Comparison of modeling approaches for constructing inverse probability weights and estimating the odds of chlamydia or gonorrhea infection given high vs. low participation in the intervention. |

There is no single best way to perform model selection. Some model selection routines require extensive programming and may only be accessible with substantial help. We recommend trying a variety of approaches and a variety of models; using different permutations of an expert knowledge model or stepwise approaches is a reasonable alternative to automated selection techniques. If results differ substantially by model selection procedure, further work is required to determine how and why the weights differ. Finally, one can derive inference using robust standard errors or techniques such as bootstrapping.^{32}

Weights should be assessed for their range – giving anyone a weight of 100 in a study with only 100 observations would mean that one person is contributing a large portion of the information towards the estimate of the parameter of interest. Extreme weights can be truncated; one can also truncate or transform variables that might be contributing to extreme or unstable weights, or one can restrict analyses to a subset of observations (see ETA assumption discussed below). One can also use so-called stabilized weights.^{17} Following sensitivity analyses for different truncation points, we chose to truncate our weights at a value of 20 to avoid over-representing any single observation or permitting extreme outliers to have very large weights.

Assumptions

The models we use require meeting a set of assumptions about the sampled data with respect to the larger population. In the case of IPW, the requisite assumptions have been laid out in detail.^{17, 26} First, one assumes there is no unmeasured confounding. There is no way to empirically test for no unmeasured confounding; collection of data on a complete set of covariates should be incorporated in the design phase. For the *Encontros* study, we collected data on a broad and inclusive set of measurable covariates.

Second, time-ordering is necessary for casual inference. Because there is no way to know exactly when someone acquires an infection and because we cannot collect samples every day, we make the assumption in this study that variables reflecting activities, behaviors, and the social environment over the past three months predate infection at the end of the period (samples were collected at the end of each interval). Additionally, we have only included participants who were treated for previous infections, such that they are at risk in every interval included in the data.

Third, Experimental Treatment Assignment (ETA) or positivity assumes that groups defined by all possible combinations of covariates must have the potential to be in any (either) of the treatment groups. If there are covariate groups that will only be observed in one treatment state, then we cannot estimate the effect of the exposure within that group. There is also the practical ETA violation, which occurs when there are too few subjects who have one of the treatments in a covariate group to estimate the treatment effect in that group. Fundamentally ETA is not a problem of the estimator, but a problem pertaining to covariate groups for which there is no experimentation and is, therefore, of issue across estimation approaches. However, most typical regression analyses will not provide obvious alerts of this problem (it is invisible to these techniques), whereas the IPW estimator can “blow-up” as weights get very large. This is an advantage of the IPW estimator, though often disappointing to the researcher because it raises a red flag about a lack of information in the data to answer the question of interest. In our data, using 3 of 4 modeling methods almost all participants had >.05 chance of being in the high participation group. One can use rules of thumb to decide the importance of the bias (such as weights > 20), though there are also established computer intensive methods to examine bias due to a potential ETA violation.^{33}