Prediction of HIV Acquisition Among Men Who Have Sex with Men

doi:10.1097/OLQ.0b013e3181a9cc41

Sex Transm Dis. Author manuscript; available in PMC 2010 September 1.

Published in final edited form as:

Sex Transm Dis. 2009 September; 36(9): 547–555.

doi: 10.1097/OLQ.0b013e3181a9cc41

PMCID: PMC2753675

NIHMSID: NIHMS126943

Prediction of HIV Acquisition Among Men Who Have Sex with Men

Timothy W Menza, BA,^1,^2,⁶ James P Hughes, PhD,^1,³ Connie L Celum, MD, MPH,^2,^4,⁵ and Matthew R Golden, MD, MPH^1,^2,^5,⁶

¹The Center for AIDS and STD, University of Washington, Seattle

²Department of Epidemiology, University of Washington, Seattle

³Department of Biostatistics, University of Washington, Seattle

⁴Department of Global Health, University of Washington, Seattle

⁵Department of Medicine, University of Washington, Seattle

⁶Public Health—Seattle & King County, Seattle, Washington

Address correspondence and requests for reprints to: Timothy W. Menza, Harborview Medical Center, 325 Ninth Avenue, Box 359931, Seattle, WA 98104, Telephone: (206) 744-3643, Facsimile: (206) 744-3693, E-mail: menza/at/u.washington.edu

The publisher's final edited version of this article is available at Sex Transm Dis

See other articles in PMC that cite the published article.

Abstract

Objective

To develop and validate an easy-to-use prediction model for HIV acquisition among men who have sex with men (MSM).

Methods

We developed prediction models using medical records data from an STD clinic (2001–2008) and validated these models using data from the control arm of Project Explore, an HIV prevention trial (1999–2003).

Results

Of 1903 MSM who tested for HIV more than once in the development sample, 101 acquired HIV over 6.7 years of follow-up. Annual HIV incidence was 2.57% (95% confidence interval [CI]: 2.09%, 3.12%). During 4 years of follow-up of 2081 Project Explore control arm participants, 144 acquired HIV for an incidence of 2.32% (95%CI: 1.96%, 2.73%). A prediction model that included variables indicating use of methamphetamine or inhaled nitrites in the prior 6 months, unprotected anal intercourse with a partner of positive or unknown HIV status in the prior year, ≥10 male sex partners in the prior year, and current diagnosis or history of bacterial sexually transmitted infection was well calibrated overall (expected-observed ratio = 1.01; 95%CI: 0.97, 1.05) and had modest discriminatory accuracy at 1 year (area under the receiver-operator characteristic curve [AUC]=0.67; 95%CI: 0.60, 0.75) and at 4 years (AUC=0.66; 95%CI: 0.61, 0.71). Over four years, cumulative incidence ranged from 3.9% to 14.3% for groups of men defined by the prediction model.

Conclusions

A new risk score was predictive of HIV acquisition and could assist providers in counseling MSM and in targeting intensified prevention to MSM at greatest risk for HIV infection. Its accuracy requires further evaluation.

Keywords: HIV, sexually transmitted infection, models/projections, risk assessment, men who have sex with men

Introduction

Men who have sex with men (MSM) continue to comprise the greatest number of HIV infections in the U.S. and many other nations.¹^–³ However, HIV risk is not evenly distributed among all MSM,⁴^–⁸ and efforts to prevent HIV might be more effectively employed if they stratified men based on validated risk criteria.

Clinical prediction models are an efficient means of identifying individuals at high risk for a medical problem who might benefit from treatment or intensified prevention efforts. Scientists have created models to predict the risk of developing coronary heart disease,⁹ diabetes,¹⁰ breast cancer,¹¹^,¹² lung cancer,¹³ and stroke among persons with atrial fibrillation,¹⁴ and findings from these models have contributed to the development of treatment and prevention guidelines for these diseases.¹⁵^–²⁰

Several studies have identified characteristics, behaviors, and contexts that associated with HIV acquisition,⁴^–⁸ but few studies have used these factors to predict the probability of future HIV acquisition.²¹^–²³ To our knowledge, no studies have created prediction models that are specific to the MSM population. Prediction models may help clinicians better counsel clients about their HIV risk, provide appropriate referrals to further services, make recommendations for the frequency of future HIV testing, and provide intensified interventions to men at highest risk. Therefore, we constructed and validated a multivariable risk score predictive of HIV acquisition among MSM.

Methods

Design Overview

We used data from Public Health—Seattle & King County (PHSKC) STD Clinic electronic records to develop a model predictive of HIV acquisition among MSM. We validated these scores using data from control group participants of Project Explore.²⁴

Risk Score Development

STD Clinic Repeat Testers

We included all MSM who tested for HIV >1 time in the PHSKC STD clinic between October 2001 and May 2008 and who tested HIV-negative by second-generation enzyme immunoassay (EIA) at their initial HIV test during the study period. Starting in 2003, all MSM negative on HIV EIA were also tested using pooled HIV RNA.²⁵ To account for the window period of HIV EIA, we excluded men from the analysis if <30 days elapsed between their first and last HIV testing visits if they were not tested using pooled RNA.

Baseline Measurement

PHSKC STD clinic providers use a structured form to record client histories before HIV testing. In a face-to-face encounter, clinicians collect information on demographic variables and self-reported substance use, STD history, and sexual behavior. Clinicians ask all clients whether they have had sex with men, women, or both in the prior year. We classified men who reported sex with another man in the prior year as MSM. For MSM, sexual behavior data include the number of male sex partners (anal and oral combined) and how often clients used condoms (always, usually, sometimes, never) during insertive and receptive anal intercourse with HIV-positive, HIV-negative, or HIV-unknown partners. These data use a 12-month recall period. In contrast, for substance use history, clinicians ask MSM whether they have ever used methamphetamine, injection drugs, inhaled nitrites, or crack and, if so, to provide the most recent month and year they used these substances. Syphilis was diagnosed using rapid plasma reagin (RPR) confirmed with Treponema pallidum particle agglutination (TPPA) assay. Urethral gonococcal and chlamydial infections were diagnosed using Aptima Combo 2 (Genprobe, Inc., San Diego, CA) or culture, while rectal and pharyngeal gonococcal and chlamydial infections were diagnosed by culture. Gonococcal cultures used modified Thayer-Martin media and chalmydial cultures used McCoy cells.

Model Building

We selected 6 binary variables on the basis of simplicity and epidemiological evidence for inclusion in the predictive model: age less than 40 years, non-white and non-Asian/Pacific Islander (API) race/ethnicity, current laboratory diagnosis of a bacterial STD (gonorrhea, Chlamydia, or early syphilis) or of having ever had a bacterial STD,⁸^,²⁸^–³⁰ use of methamphetamine or inhaled nitrites in the prior 6 months,⁴^,⁷^,²⁹^,³¹^,³² ≥10 male sex partners in the prior year,⁴^,³² and unprotected anal intercourse with a partner of unknown or positive HIV status (non-concordant UAI) in the prior year.⁴^,³³ This model is the full model.

We considered the inclusion of herpes simplex virus type-2 (HSV-2) infection given its high prevalence among MSM and its association with HIV acquisition;³⁴ however, excluded it because only 10% of men were tested for HSV-2 at their first HIV testing visit and only 8% reported a history of HSV-2 infection. Despite the different physiological effects of methamphetamine and inhaled nitrites, we grouped methamphetamine and inhaled nitrite use into one variable for simplicity and to facilitate uptake of the prediction models by clinicians. Additionally, these substances are used in similar contexts³⁵^–³⁹ and the magnitude of associations between HIV acquisition and these substances are similar.⁴^,⁷^,²⁹^,³¹^,³² We used a 6-month recall period for methamphetamine and inhaled nitrite use to be consistent with previous reports and to improve the reliability of the measures over those using longer recall periods.⁴^,⁷^,²⁹^,³¹^,³² We performed analyses with separate variables for methamphetamine and inhaled nitrites use and analyses with a 12-month recall period for the composite variable; the results were very similar to those in this manuscript and are not presented.

Because one’s age and race/ethnicity are immutable characteristics, we tested a second, simple model that excluded the variables indicating age and race/ethnicity. We also used a step-wise selection procedure using the Akaike Information Criterion (AIC) to create a more parsimonious model from the initial 6-variable model.²⁴^,⁴⁰^,⁴¹ This procedure yielded a model that included all the predictors except for non-concordant UAI in the prior year. Because non-concordant UAI was the strongest predictor of incident HIV infection in a cross-sectional analysis,³³ we kept it in our prediction model and did not evaluate a model without this predictor.

We used Cox proportional hazards models to estimate hazard ratios (HR) for HIV acquisition. Because clinicians and disease intervention specialists concentrate on current reports of risk and diagnoses of STD in counseling clients, the values of the predictors in the models were those at baseline; predictors were not modeled as time-varying covariates. The initial time point for survival modeling was the date of first negative HIV EIA. For MSM who tested HIV-positive by reactive EIA, we defined time-of-acquisition as the midpoint between the date of their positive EIA and the date of their last negative EIA. For men who tested HIV-positive by pooled RNA screening but HIV-negative by EIA at the same visit, we defined time-of-acquisition as the date of the positive RNA test. MSM were censored at their last HIV testing visit if they had not tested HIV-positive in the STD Clinic by 12 May 2008. Additionally, we repeated model building and development using the predictor values recorded at clients’ second visits at which they tested HIV-negative; the results of those analyses were similar to those using clients’ first visits and are not presented here.

For the full and simple models, all predictors met the proportional hazards assumption as assessed with log(-log) plots of the survival function, Schoenfeld residuals and associated test statistics,⁴² and including interaction terms with time for each predictor variable. Additionally, we found no evidence for interactions between individual risk predictors assessed by a global likelihood ratio test.

Based on the coefficients from the Cox proportional hazards regression models, we created integer weights for each variable. We calculated these weights by multiplying the model coefficients by 10 (the choice of multiplier is arbitrary) and rounding to the nearest whole integer. The weights rank the risk predictors in relative importance and dictate how one assigns integer point values for each risk predictor for a given individual. The assigned points are then summed to compute that individual’s risk score.

We also performed interval-censored analyses assuming a log-logistic distribution to model the odds of HIV acquisition associated with each risk predictor.⁴³^,⁴⁴ These methods yielded similar results to the Cox proportional hazards models presented here.

Stability of Simple Model Predictors

To assess the consistency of risk among men evaluated in our clinic on multiple occasions, we used data collected from men on their first visit to the clinic each year for the first 4 years of follow-up to calculate kappa statistics for the variables in the simple model.⁴⁵

Risk Score Validation

Project Explore

The control arm of Project Explore comprised the population for model validation. Project Explore was a randomized behavioral intervention trial that enrolled HIV-negative MSM age >16 from six U.S. cities, including Seattle, between January 1999 and February 2001. Men were eligible if they reported having anal sex with ≥1 men in the past year, but were excluded if they reported a mutually monogamous relationship lasting ≥2 years with an HIV-negative male partner. At enrollment and every six months thereafter, Explore participants completed an audio computer-assisted self-interview (ACASI) to assess sexual behavior, substance use, and recent STD diagnoses in the prior 6 months, and were tested for HIV by EIA. Project Explore followed participants for 48 months or until January 2003, whichever came first, and 87% of men in the control arm completed the study.⁴⁶ The variables in Project Explore that corresponded to the predictors selected in the STD clinic sample were: methamphetamine or inhaled nitrites use in the prior 6 months; non-concordant UAI in the prior 6 months; bacterial STD in the prior 6 months; and, ≥10 sex partners in the prior 6 months. All of these variables are based on participant self-report.

Analysis

Using the full and simple models derived from the STD clinic sample, we calculated two risk scores for each man in the control arm of Project Explore based on baseline assessments. We calculated the annual incidence of HIV infection for each quartile of risk score. We assessed model calibration by calculating the ratio of expected HIV infections to observed HIV infections (expected-observed ratio),¹² where calibration is defined as the extent to which the risk predicted by the model reflects the risk in the observed in population.⁴⁷ The 95% confidence intervals for HIV incidence and the expected-observed ratio assume that the observed HIV infections follow a Poisson distribution. We also assessed model calibration graphically by comparing plots of observed and predicted survival estimates for each risk group.²⁴

To assess the ability of the risk scores to discriminate between men who acquired HIV and those who did not, we calculated the area under time-dependent receiver operating characteristic curves (AUC) for “survival” at one year of follow-up and at four years of follow-up with nonparametric bootstrap 95% confidence intervals using the methods of Heagerty and colleagues.⁴⁸^,⁴⁹

All analyses were conducted using STATA 10.0 (StataCorp, College Station, TX) and R 2.6.1 (The R Project for Statistical Computing). The University of Washington Human Subjects Division approved all study procedures.

Results

Study populations and HIV incidence

Between October 2001 and May 2008, PHSKC STD clinic staff evaluated 6026 MSM who did not acknowledge previously testing HIV-positive; 290 (5%) of these men tested HIV-positive and 5736 (95%) men tested HIV-negative. Of these 5736 HIV-negative men, 3827 (67%) did not return for HIV testing during the study period and 1909 (33%) returned at least once for HIV testing (median 3 visits, range 2–28), and 1903 men tested for HIV again ≥30 days after their initial EIA and comprise our sample for model development.

Compared to men who tested for HIV only once during the study period, at baseline repeat testers were more likely to be white (68% v. 71%, P=0.01) and <40 years of age (70% v. 80%, P<0.001), but less likely to use crack or cocaine (5% v. 3%, P<0.001) or injection drugs (4% v. 2%, P<0.001) in the prior 6 months. Men who did not return to test were less likely than men who tested repeatedly to test positive for bacterial STD (12% v. 20%, P<0.001) at their only or first visits, respectively, and to report having ever been diagnosed with a bacterial STD (28% v. 33%, P<0.001). Men who did and did not return for testing reported similar rates of methamphetamine and inhaled nitrite use in the prior 6 months and of non-concordant UAI in the prior year.

Repeat HIV testers in the STD clinic sought testing over a median of 1.6 years (range: 30 days – 6.7 years) and experienced 101 HIV infections (annual incidence = 2.57%; 95%CI: 2.09%, 3.12%). Of the 2081 Explore control group participants, 144 acquired HIV over a median of 3 years (range: 6 months – 4 years) for an annual incidence of 2.32% (95%CI: 1.96%, 2.73%).

Explore participants were older, more likely to be Hispanic, and reported greater rates of partner change than STD clinic clients (Table 1). Explore participants were also more likely to report using inhaled nitrites and methamphetamine in the preceding 6 months than STD clinic testers. MSM tested at the STD clinic were more likely to be API than Explore participants, and more likely to be diagnosed with a bacterial STD at their first HIV testing visit than Explore participants in the 6 months preceding their enrollment interview. Fewer STD clinic clients reported having non-concordant UAI in the preceding 12 months than Explore participants did in the prior 6 months.

Table 1

Baseline socio-demographic characteristics and sexual and drug use behavior of Public Health—Seattle & King County repeat HIV testers (2001–2008) and Project Explore control arm participants (1999–2003).

Agreement of simple model risk predictors within an individual over time

Certain variables may be poor predictors of HIV infection if they are intermittent as opposed to consistent. For MSM tested at the STD clinic, we evaluated the agreement between the predictors of the simple model each year over the first 4 years of follow-up. The kappa statistics were 0.41 for use of methamphetamine or inhaled nitrites in the prior 6 months, 0.47 for ≥10 male sex partners in the prior year, 0.28 for non-concordant UAI in the prior year, and 0.67 for history, or current diagnosis, of bacterial STD.

Risk score evaluation and validation

We used the full and simple prediction models to generate scores for each participant in the control arm of Project Explore based on their baseline assessments (Table 2). The mean (standard deviation [SD]) scores for the full and simple models were 11.7 (6.9) and 5.6 (6.0), respectively; the median (range) scores were 11 (0–31) and 3 (0–19), respectively. The incidence of HIV and the hazard ratio for HIV infection increased with quartile of risk score (test of trend P < 0.0001 for each index; Table 3). For groups of men who differ by one unit in the full risk score, men with the higher risk score experience a risk of HIV 1.08 times greater than that experienced by men with the lower risk score (95% CI: 1.06, 1.11, P<0.001). For groups of men who differ by one category (quartile) of the full risk score, men in the higher category experience a risk of HIV 1.61 times greater than that experience by men in the lower category (95%CI: 1.39, 1.86; P<0.001). Comparable hazard ratios for the simple model were similar. The full and simple models were well calibrated in the validation sample (Table 4). The expected annual rate of HIV infection was 2.35% according to the full model (expected-observed ratio, 1.01; 95%CI: 0.97, 1.06) and 2.34% according to the simple model (expected-observed ratio, 1.01; 95%CI: 0.97, 1.05). Inspection of plots comparing Kaplan-Meier survival curves and Cox proportional hazards survival curves for each risk quartile (data not shown), also indicated acceptable model calibration.

Table 2

Multivariable prediction models for HIV acquisition among MSM developed in Public Health—Seattle & King County repeat HIV testers, 2001–2008.

Table 3

Application of the full and simple MSM HIV acquisition prediction models to Project Explore control arm participants (1999–2003).

Table 4

Overall calibration and discrimination of the full and simple MSM HIV acquisition prediction models applied to Public Health—Seattle & King County repeat STD clinic testers (2001–2008) and Project Explore control arm participants (more ...)

Model discrimination measured by time-dependent AUC estimates was 0.68 (95%CI: 0.62, 0.75) at one year and 0.67 (95%CI: 0.62, 0.71) at four years for the full model in Project Explore (Table 4). The simple model exhibited similar discrimination, 0.67 (95%CI: 0.60, 0.75) at one year and 0.66 (95%CI: 0.61, 0.71) at four years. Because our results may be influenced by temporal trends in sexual risk, substance use, and STI diagnoses over time and by the potential inclusion of Seattle Explore participants in the development sample, we performed two sensitivity analyses. The first assessed the impact of restricting model development and validation to the years 2001–2003 during which the STD clinic and Explore study periods overlapped. The second excluded the 371 Seattle participants from the Project Explore control arm on model discrimination. Time-dependent AUC estimates at one and four years were similar in magnitude and not different statistically from our original estimates (data not shown).

Because the full and simple models performed similarly based on their calibration and discrimination, Table 5 presents the sensitivity, specificity, and cumulative HIV incidence for groups of men defined by the score cut-offs for the simple model over 1 year and 4 years of follow-up. The Figure displays the cumulative incidence of HIV infection over time for each quartile of the simple risk score in Project Explore. Cumulative incidence at four years was 3.9% (95%CI: 2.4%, 6.2%) in men with a score of zero, 5.3% (95%CI: 2.1%, 8.5%) in men with scores 1–3, 9.5% (95%CI: 4.6%, 14.4%) in men with scores 4–11, and 14.3% (95%CI: 7.5%, 21.1%) in men with scores ≥12.

Table 5

Sensitivity, specificity, and cumulative incidence of HIV acquisition for risk scores defined by the simple model, Public Health—Seattle & King County repeat testers (2001–2008) and Project Explore control arm (1999–2003). (more ...)

Figure

Cumulative incidence of HIV infection the control arm of Project Explore, 1999–2003, according to risk quartiles defined by the simple prediction model developed among MSM seen in the Public Health—Seattle & King County STD Clinic, (more ...)

Discussion

We used clinical and behavioral data collected from an STD clinic to develop two risk scores predictive of HIV infection, and validated those scores using data from a large multi-center behavioral intervention trial. While the scores we developed would optimally undergo additional validation in other populations,⁵⁰^,⁵¹ we believe our simple score can be useful to clinicians and others in counseling MSM about their risk of HIV infection, and might be used to identify persons who require intensified interventions or more frequent HIV testing. The resources necessary to conduct biomedical and behavioral HIV prevention trials depend on HIV incidence.⁴⁶^,⁵²^,⁵³ Thus, this score might also help researchers identify MSM at high risk of HIV infection for prevention trials with HIV acquisition as the primary outcome.

Despite the model’s simplicity and excellent calibration, its discriminatory accuracy was only modest (AUC, 0.66). However, this finding should be interpreted with the understanding that extremely high relative risks (>100) are required to generate risk predictor models with high AUC estimates,¹²^,⁵⁴^–⁵⁶ and that such models are extremely uncommon in clinical practice. The AUC estimates that we report are similar to those of other risk models, such as the Framingham risk score for coronary heart disease (AUC, 0.63 to 0.83),⁵⁷ that are commonly used to guide clinical decisions.

While the calculation of the risk scores may seem daunting, the scores we present are not more complicated than the Framingham risk score⁹ or the CHADS score¹⁴, tools that clinicians commonly use to estimate patients’ risk of coronary heart disease and the risk of stroke among persons with atrial fibrillation, respectively. Our MSM risk score should not require the use of a computer for calculation, and may be suitable for use in non-clinical settings or on the Internet. We are currently developing a public health website that will allow MSM to calculate their score and explore their risk of HIV acquisition. An approach to calculating the simple risk score and estimating a man’s 4-year risk of acquiring HIV is presented in the Box.

Box

Estimating a patient’s 4-year risk of HIV acquisition according to the simple model.

The ideal risk score cut-off may be one that leads to the follow-up of a number of patients that the resources of a clinic, provider, or community-based organization can accommodate. We found that using a score that identifies men with any risk predictor (i.e., risk score ≥1), our simple model would identify 83% of STD clinic patients and 86% of Explore participants who acquired HIV over 4 years, but would require follow-up of approximately 70% of the two populations. Using a higher risk score identifies a population at greater risk, but a smaller proportion of all men who acquire HIV. Currently, our STD clinic employs the simple model to identify men with ≥1 risk predictor for more frequent HIV testing, follow-up counseling by telephone, and reminder notices to return for repeat HIV testing.

The strongest predictors of HIV acquisition in our models were use of methamphetamine or inhaled nitrites and a history, or current diagnosis, of bacterial STD, findings that highlight the importance of routinely asking MSM about these drugs and of concentrating prevention efforts in persons with bacterial STD. While non-concordant UAI has typically been the strongest risk factor for testing HIV-positive in cross-sectional studies,³³ it was a weaker predictor of HIV acquisition in our models. This difference may reflect the distinction between behaviors or characteristics that operate at the individual-level and those that operate at a network-level. For example, substance use and STD may indicate that an individual circulates within a sexual network more conducive to HIV transmission, while the report of non-concordant UAI does not always indicate this network-level risk. Additionally, we found that the agreement in reports of non-concordant UAI over four years of follow-up was only fair. This fair agreement likely diminishes our predictive models’ discriminatory ability. We overestimate the risk of HIV acquisition for men who report non-concordant UAI at baseline, but do not practice non-concordant UAI during the follow-up period. Conversely, we may underestimate the risk of HIV acquisition for men who do not report non-concordant UAI at baseline, but engage in non-concordant UAI during the follow-up period.

Our study is subject to important limitations. First, our risk score was derived from a population of MSM who repeatedly tested in a single U.S. STD clinic. Not all MSM in our clinic tested serially during the study period, and studies suggest that risk behavior may influence HIV testing behavior.⁵⁸^,⁵⁹ On the other hand, our models performed reasonably well among MSM from six U.S. cities who tested at 6-month intervals as part of Project Explore, suggesting that the prediction models can be applied to a more diverse population of MSM. However, as our development and validation samples were composed mostly of white MSM, we cannot be sure of our models’ performance in racial and ethnic minority MSM. Additionally, we cannot be sure of our models’ applicability to populations and geographic areas with lower rates of methamphetamine and inhaled nitrites use.

Second, data collected from the PHSKC STD Clinic and Project Explore were dissimilar in important ways. Project Explore employed ACASI and collected behavioral and STD diagnosis data for 6-month periods, while clinicians at the PHSKC STD Clinic collected behavioral data through face-to-face interviews, and sexual behavior questions focused on the prior 12 months. Also, data concerning bacterial STD were based on self-report and diagnoses on the day of HIV testing in the STD clinic, but reflected only self-reported history of bacterial STD in the prior 6 months in Project Explore. Misclassification would diminish model discrimination. For example, men in the STD clinic who practiced non-concordant UAI might not acknowledge this behavior because of social desirability bias associated with face-to-face interviews. Therefore, the predictive models would overestimate the risk of HIV acquisition in men who did not report non-concordant UAI in both the development and validation samples.

Prediction models are improved through repeated validation, augmentation, and examination in research and practice.⁹^,¹⁵^,¹⁷ We present our prediction models as a step in the process of developing consistent guidelines for HIV prevention practice. Optimally, future work should augment and validate these prediction models in diverse settings and measure the acceptability and utility of prevention efforts guided by prediction models.⁵⁰^,⁵¹ However, even without such additional research, we believe our prediction model can be useful in counseling MSM and in prioritizing intensified prevention efforts to the MSM at highest risk for HIV.

Acknowledgments

The authors thank Deborah Donnell and Marla Husnik from the Statistical Center for HIV/AIDS Research and Prevention and Roxanne Kerani and Fred Koch from Public Health—Seattle & King County for their technical assistance.

Financial support. Project Explore was supported by the HIV Prevention Trials Network and sponsored by the National Institutes of Health and the U.S. Office of AIDS Research. Project Explore was not conducted independent of the funding sources. However, funding organizations had no involvement in the present study’s design; data collection, analysis, or interpretation; or in the decision to publish this manuscript. TWM was supported by NIH T32 AI07140.

Footnotes

Potential conflicts of interest. TWM, no conflict; JPH, no conflict; CLC, no conflict; MRG, no conflict.

References

1. Subpopulation estimates from the HIV incidence surveillance system--United States, 2006. MMWR Morb Mortal Wkly Rep. 2008 Sep 12;57(36):985–989. [PubMed]

2. Hall HI, Song R, Rhodes P, et al. Estimation of HIV incidence in the United States. Jama. 2008 Aug 6;300(5):520–529. [PMC free article] [PubMed]

3. Baral S, Sifakis F, Cleghorn F, et al. Elevated risk for HIV infection among men who have sex with men in low- and middle-income countries 2000–2006: a systematic review. PLoS Med. 2007 Dec;4(12):e339. [PMC free article] [PubMed]

4. Buchbinder SP, Vittinghoff E, Heagerty PJ, et al. Sexual risk, nitrite inhalant use, and lack of circumcision associated with HIV seroconversion in men who have sex with men in the United States. J Acquir Immune Defic Syndr. 2005 May 1;39(1):82–89. [PubMed]

5. Koblin BA, Husnik MJ, Colfax G, et al. Risk factors for HIV infection among men who have sex with men. Aids. 2006 Mar 21;20(5):731–739. [PubMed]

6. Hall HI, Byers RH, Ling Q, et al. Racial/ethnic and age disparities in HIV prevalence and disease progression among men who have sex with men in the United States. Am J Public Health. 2007 Jun;97(6):1060–1066. [PubMed]

7. Plankey MW, Ostrow DG, Stall R, et al. The Relationship Between Methamphetamine and Popper Use and Risk of HIV Seroconversion in the Multicenter AIDS Cohort Study. J Acquir Immune Defic Syndr. 2007 Feb 22; [PMC free article] [PubMed]

8. Schwarcz SK, Kellogg TA, McFarland W, et al. Characterization of sexually transmitted disease clinic patients with recent human immunodeficiency virus infection. J Infect Dis. 2002 Oct 1;186(7):1019–1022. [PubMed]

9. Lloyd-Jones DM, Larson MG, Beiser A, et al. Lifetime risk of developing coronary heart disease. Lancet. 1999 Jan 9;353(9147):89–92. [PubMed]

10. Griffin SJ, Little PS, Hales CN, et al. Diabetes risk score: towards earlier detection of type 2 diabetes in general practice. Diabetes Metab Res Rev. 2000 May–Jun;16(3):164–171. [PubMed]

11. Gail MH, Brinton LA, Byar DP, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989 Dec 20;81(24):1879–1886. [PubMed]

12. Tice JA, Cummings SR, Smith-Bindman R, et al. Using clinical factors and mammographic breast density to estimate breast cancer risk: development and validation of a new predictive model. Ann Intern Med. 2008 Mar 4;148(5):337–347. [PMC free article] [PubMed]

13. Cassidy A, Duffy SW, Myles JP, et al. Lung cancer risk prediction: a tool for early detection. Int J Cancer. 2007 Jan 1;120(1):1–6. [PubMed]

14. Gage BF, Waterman AD, Shannon W, et al. Validation of clinical classification schemes for predicting stroke: results from the National Registry of Atrial Fibrillation. Jama. 2001 Jun 13;285(22):2864–2870. [PubMed]

15. Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation. 2002 Dec 17;106(25):3143–3421. [PubMed]

16. Bach PB, Niewoehner DE, Black WC. Screening for lung cancer: the guidelines. Chest. 2003 Jan;123(1 Suppl):83S–88S. [PubMed]

17. Chobanian AV, Bakris GL, Black HR, et al. Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension. 2003 Dec;42(6):1206–1252. [PubMed]

18. Qaseem A, Snow V, Sherif K, et al. Screening mammography for women 40 to 49 years of age: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2007 Apr 3;146(7):511–515. [PubMed]

19. Fuster V, Ryden LE, Cannom DS, et al. ACC/AHA/ESC 2006 Guidelines for the Management of Patients with Atrial Fibrillation: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the European Society of Cardiology Committee for Practice Guidelines (Writing Committee to Revise the 2001 Guidelines for the Management of Patients With Atrial Fibrillation): developed in collaboration with the European Heart Rhythm Association and the Heart Rhythm Society. Circulation. 2006 Aug 15;114(7):e257–e354. [PubMed]

20. Nutrition Recommendations and Interventions for Diabetes: a position statement of the American Diabetes Association. Diabetes Care. 2007 Jan;30 Suppl 1:S48–S65. [PubMed]

21. Boileau C, Bruneau J, Al-Nachawati H, et al. A prognostic model for HIV seroconversion among injection drug users as a tool for stratification in clinical trials. J Acquir Immune Defic Syndr. 2005 Aug 1;39(4):489–495. [PubMed]

22. Chen Z, Branson B, Ballenger A, et al. Risk assessment to improve targeting of HIV counseling and testing services for STD clinic patients. Sex Transm Dis. 1998 Nov;25(10):539–543. [PubMed]

23. Richert CA, Peterman TA, Zaidi AA, et al. A method for identifying persons at high risk for sexually transmitted infections: opportunity for targeting intervention. Am J Public Health. 1993 Apr;83(4):520–524. [PubMed]

24. Harrell FE, Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996 Feb 28;15(4):361–387. [PubMed]

25. Stekler J, Swenson PD, Wood RW, et al. Targeted screening for primary HIV infection through pooled HIV-RNA testing in men who have sex with men. Aids. 2005 Aug 12;19(12):1323–1325. [PubMed]

26. Slide Set: HIV/AIDS Surveillance in Men Who Have Sex with Men (MSM) [Accessed November 21, 2008]. Available at: http://www.cdc.gov/hiv/topics/surveillance/resources/slides/msm/index.htm.

27. Choi KH, McFarland W, Neilands TB, et al. An opportunity for prevention: prevalence, incidence, and sexual risk for HIV among young Asian and Pacific Islander men who have sex with men, San Francisco. Sex Transm Dis. 2004 Aug;31(8):475–480. [PubMed]

28. Wasserheit JN. Epidemiological synergy. Interrelationships between human immunodeficiency virus infection and other sexually transmitted diseases. Sex Transm Dis. 1992 Mar–Apr;19(2):61–77. [PubMed]

29. Page-Shafer K, Veugelers PJ, Moss AR, et al. Sexual risk behavior and risk factors for HIV-1 seroconversion in homosexual men participating in the Tricontinental Seroconverter Study, 1982–1994. Am J Epidemiol. 1997 Oct 1;146(7):531–542. [PubMed]

30. Truong HM, Grant RM, McFarland W, et al. Routine surveillance for the detection of acute and recent HIV infections and transmission of antiretroviral resistance. Aids. 2006 Nov 14;20(17):2193–2197. [PubMed]

31. Chesney MA, Barrett DC, Stall R. Histories of substance use and risk behavior: precursors to HIV seroconversion in homosexual men. Am J Public Health. 1998 Jan;88(1):113–116. [PubMed]

32. Burcham JL, Tindall B, Marmor M, et al. Incidence and risk factors for human immunodeficiency virus seroconversion in a cohort of Sydney homosexual men. Med J Aust. 1989 Jun 5;150(11):634–639. [PubMed]

33. Golden MR, Brewer DD, Kurth A, et al. Importance of sex partner HIV status in HIV risk assessment among men who have sex with men. J Acquir Immune Defic Syndr. 2004 Jun 1;36(2):734–742. [PubMed]

34. Brown EL, Wald A, Hughes JP, et al. High risk of human immunodeficiency virus in men who have sex with men with herpes simplex virus type 2 in the EXPLORE study. Am J Epidemiol. 2006 Oct 15;164(8):733–741. [PubMed]

35. Shoptaw S, Reback CJ, Freese TE. Patient characteristics, HIV serostatus, and risk behaviors among gay and bisexual males seeking treatment for methamphetamine abuse and dependence in Los Angeles. J Addict Dis. 2002;21(1):91–105. [PubMed]

36. Romanelli F, Smith KM, Thornton AC, et al. Poppers: epidemiology and clinical management of inhaled nitrite abuse. Pharmacotherapy. 2004 Jan;24(1):69–78. [PubMed]

37. Colfax GN, Mansergh G, Guzman R, et al. Drug use and sexual risk behavior among gay and bisexual men who attend circuit parties: a venue-based comparison. J Acquir Immune Defic Syndr. 2001 Dec 1;28(4):373–379. [PubMed]

38. Reback CJ. The Social Construction of a Gay Drug: Methamphetamine Use Among Gay and Bisexual Males in Los Angeles. Los Angeles, CA: 1997.

39. Schwarcz S, Scheer S, McFarland W, et al. Prevalence of HIV infection and predictors of high-transmission sexual risk behaviors among men who have sex with men. Am J Public Health. 2007 Jun;97(6):1067–1075. [PubMed]

40. Akaike H. Information theory and an extension of the maximum likelihood principle. In: Petrov B, Csaki F, editors. Second International Symposium on Inference Theory. Budapest: Akademiai Kiado; 1973. pp. 267–281.

41. Hosmer DW, Lemeshow S. Applied Survival Analysis. New York: Wiley; 1999.

42. Grambsch P, Therneau T. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika. 1994;(81):515–526.

43. Griffin JT, Fraser C, Gras L, et al. The effect on treatment comparisons of different measurement frequencies in human immunodeficiency virus observational databases. Am J Epidemiol. 2006 Apr 1;163(7):676–683. [PubMed]

44. Lindsey JC, Ryan LM. Tutorial in biostatistics methods for interval-censored data. Stat Med. 1998 Jan 30;17(2):219–238. [PubMed]

45. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20:37–46.

46. Koblin B, Chesney M, Coates T. Effects of a behavioural intervention to reduce acquisition of HIV infection among men who have sex with men: the EXPLORE randomised controlled study. Lancet. 2004 Jul 3–9;364(9428):41–50. [PubMed]

47. Harrell FE., Jr Regression Modelling Strategies: With Applications to Linear Models, Logisitic Regression, and Survival Analysis. New York: Springer-Verlag; 2001.

48. Heagerty PJ, Lumley T, Pepe MS. Time-dependent ROC curves for censored survival data and a diagnostic marker. Biometrics. 2000 Jun;56(2):337–344. [PubMed]

49. Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 2005 Mar;61(1):92–105. [PubMed]

50. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999 Mar 16;130(6):515–524. [PubMed]

51. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006 Feb 7;144(3):201–209. [PubMed]

52. Kamb ML, Fishbein M, Douglas JM, Jr, et al. Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases: a randomized controlled trial. Project RESPECT Study Group. Jama. 1998 Oct 7;280(13):1161–1167. [PubMed]

53. Imrie J, Stephenson JM, Cowan FM, et al. A cognitive behavioural intervention to reduce sexually transmitted infections among gay men: randomised trial. BMJ. 2001 Jun 16;322(7300):1451–1456. [PMC free article] [PubMed]

54. Pepe MS, Janes H, Longton G, et al. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004 May 1;159(9):882–890. [PubMed]

55. Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ. 1999 Dec 11;319(7224):1562–1565. [PMC free article] [PubMed]

56. Ware JH. The limitations of risk factors as prognostic tools. N Engl J Med. 2006 Dec 21;355(25):2615–2617. [PubMed]

57. D'Agostino RB, Sr, Grundy S, Sullivan LM, et al. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001 Jul 11;286(2):180–187. [PubMed]

58. MacKellar DA, Valleroy LA, Secura GM, et al. Repeat HIV testing, risk behaviors, and HIV seroconversion among young men who have sex with men: a call to monitor and improve the practice of prevention. J Acquir Immune Defic Syndr. 2002 Jan 1;29(1):76–85. [PubMed]

59. Kellerman SE, Lehman JS, Lansky A, et al. HIV testing within at-risk populations in the United States and the reasons for seeking or avoiding HIV testing. J Acquir Immune Defic Syndr. 2002 Oct 1;31(2):202–210. [PubMed]