Suppl 1: Developing a Web-Based HIV Behavioral Surveillance Pilot Project Among Men Who Have Sex with Men

doi:10.2174/1874613601206010224

Open AIDS J. 2012; 6: 224–231.

Published online 2012 October 5. doi: 10.2174/1874613601206010224

PMCID: PMC3474941

Suppl 1

Developing a Web-Based HIV Behavioral Surveillance Pilot Project Among Men Who Have Sex with Men

Travis Sanchez,^* Amanda Smith, Damian Denson, Elizabeth DiNenno, and Amy Lansky

Centers for Disease Control and Prevention, 1600 Clifton Road NE, Atlanta, Georgia, USA

^*Address correspondence to this author at the Emory University, Rollins School of Public Health, 1518 Clifton Rd NE, Atlanta, Georgia, USA; Tel: 404-727-8403; Fax: 404-727-8737; E-mail: Travis.Sanchez/at/emory.edu

Received April 15, 2011; Revised June 24, 2011; Accepted July 1, 2011.

This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

This article has been cited by other articles in PMC.

Abstract

Background:

A web-based HIV behavioral surveillance system (WHBS) has potential to collect behavioral data from men who have sex with men (MSM) not reached through traditional sampling methods. Six U.S. cities conducted a WHBS pilot in 2005-2007 to determine the feasibility to conduct a behavioral surveillance project entirely on the internet.

Methods:

Three sampling methods of adult MSM on the internet were explored: direct marketing (DM) using banner advertisements; respondent-driven sampling (RDS) using peer recruitment; and venue-based sampling (VBS) using internet venues.

Results:

A total of 8,434 complete MSM surveys were obtained: 8,109 through DM, 130 through RDS, and 195 through VBS. By methods, enrollment rates ranged from 70-90%; completion rates ranged from 67-95%. DM obtained the largest proportions of racial/ethnic minority MSM (36%) and MSM 18-20 years (19%).

Conclusions:

Only the DM method achieved a substantial number of complete MSM surveys. Successful implementation of an internet-based systematic sampling method may be problematic, but a convenience sample of MSM using banner advertisements is feasible and may produce useful and timely behavioral information from a large number of MSM.

Keywords: MSM, gay, internet, HIV, sexual behavior.

INTRODUCTION

The Centers for Disease Control and Prevention (CDC) released the HIV Prevention Strategic Plan in 2000 with a goal to reduce new HIV infections in the US. Objectives to reach this goal included strengthening national capacity to monitor the HIV epidemic to better direct and evaluate prevention efforts [1]. In an effort to meet this objective, CDC awarded funds to state and local health departments to develop and implement a new surveillance system, the National HIV Behavioral Surveillance System (NHBS). NHBS is an ongoing surveillance system initiated in 2003 to monitor trends in HIV-related risk behaviors [2]. From 2003-2005, NHBS focused on data collection among men who have sex with men (MSM) in 17 metropolitan statistical areas (MSAs) in the U.S. [3].

The sampling method for NHBS among MSM uses physical venues, such as bars, clubs or other locations, where MSM are approached and consented to an in-person interview [4]. NHBS investigators hypothesized that some MSM of interest for behavioral surveillance may not be reached through this NHBS sampling method, in particular, those MSM who do not attend physical venues and who use the internet for socialization and meeting sex partners. Past studies specific to MSM suggest that internet usage rates for this group are higher than for other men, and approximately 40% of MSM have reported seeking sex partners on the internet [5-8]. Some research has also identified high rates of sexual risk behavior among MSM who meet sexual partners on the internet, and sexual risk behavior may be more common among these MSM than those who meet partners elsewhere [8]. Internet use to find sex partners has also been implicated in the transmission of sexually transmitted diseases (STDs) [9-11].

A web-based HIV behavioral surveillance system (WHBS) pilot project was initiated in 2005 to examine whether the internet may be used to collect behavioral data from MSM populations that may not be sampled through NHBS. The specific objectives of the WHBS pilot project were to 1) develop and field-test methods for conducting WHBS to determine the feasibility of conducting this type of project entirely on the internet; 2) evaluate the potential of recruiting MSM not otherwise easily reached through sampling at physical venues; and 3) compare HIV-related behavioral risks between MSM reached over the internet with MSM reached through NHBS to determine if these populations have similar or different HIV infection risks.

Internet-based behavioral research is an emerging field, but several researchers have successfully implemented online behavioral research projects using a variety of convenience sampling methods, some of which have focused on HIV and MSM [5,7,12-16]. Though not meeting the stringent standards of probability-based sampling methods, systematic sampling methods for MSM on the internet may be feasible and hold additional benefits for use in a behavioral surveillance system where methods consistency and representativeness are useful attributes. The goals of this report are to describe the development and piloting of several WHBS sampling methods (both systematic and convenience), discuss outcomes related to the first objective of WHBS, and determine the feasibility of an entirely internet-based method for HIV behavioral surveillance of MSM. Assessing the comparability or complementary nature of this WHBS behavioral data to other sources, such as NHBS, is related to the second and third objectives of the WHBS pilot and is reported in another paper in this journal issue [17].

METHODS

The WHBS pilot consisted of a period of formative research activities in 2005 (data not presented) followed by two rounds of recruitment and online surveys that occurred in 2006 and 2007. WHBS was implemented in 6 MSAs: Boston, MA; Baltimore, MD; Dallas, TX; Los Angeles, CA; New York City, NY; and San Francisco, CA.

WHBS Pilot Round 1

Sampling Overview

Three cross-sectional sampling methods were utilized for this round: 1) direct marketing using banner advertisements on websites; 2) respondent-driven sampling using peer recruitment; and 3) venue-based sampling using internet “venues”. Each MSA conducted at least one method from among these three. Data collection for WHBS Round 1 occurred during an 8-week period in 2006; each WHBS project area had a sample size goal of 500 MSM with complete surveys. In order to be eligible, men must have been at least 18 years of age and a current resident of one of the 6 MSAs. Additionally, for the respondent-driven sampling eligibility, men must have had sex with a man in the past year and reported using the internet to either “get information on topics of personal interest” or “meet sexual partners or friends” in the past year. Participant reimbursements/incentives were not used. Sampling methods are described below; recruitment statistics are presented in Table 11.

Table 1.

Recruitment Rates of the Web-Based HIV Behavioral Surveillance System, 6 U.S. Metropolitan Statistical Areas, 2006-2007

Direct Marketing Sampling

Direct marketing (DM) is a convenience sampling method and consisted of banner advertisements (banner ads) displayed on a variety of websites. This method has been used for internet-based recruitment of MSM by several other researchers [5,7,13,16]. Round 1 banner ads were developed by local investigators during formative research, contained imagery and language thought to be attractive to local MSM, included limited information about the study, and allowed interested individuals direct access to the WHBS web site. The selection of web sites for banner ad placement was based on formative research findings and involved identification of sites with relevance to the local MSM population and those sites that were willing to post the WHBS banner ads. Upon clicking on the banner ad, the potential participant was linked to the WHBS web site and a unique study identification number (ID) was automatically generated to allow access. Screening for eligibility criteria occurred upon entry to the WHBS web site. Round 1 DM was conducted in 4 MSAs: Baltimore, Boston, New York City, and San Francisco.

Respondent-Driven Sampling

Respondent-driven sampling (RDS) is a systematic chain-referral sampling method that has been used in other HIV studies and may be particularly well suited to recruit individuals from hard-to-reach populations [18,19]. Briefly, RDS begins with a small number of initial participants, called “seeds”, who are purposively selected by study staff. All participants who complete the survey are asked to recruit a limited number of their peers. Recruited peers are subsequently asked to recruit additional peers, and so on, producing recruitment chains or waves. Numbering systems are used to link recruiters and their recruits.

To recruit seeds, WHBS staff utilized their own expertise and formative research feedback to develop a list of internet chat room or personal profile websites that local MSM visited. Staff used instant messages and emails to interact with potential participants. Persons who agreed to take the survey and met eligibility criteria received a unique study ID and a link to the secure WHBS web site where eligibility screening was repeated. After a seed completed the survey, he was instructed in how to recruit internet-using MSM peers for the study and received 3 unique study ID numbers and a link to the WHBS web site that he was asked to give to his peers. Peers used the study ID and link to enter the WHBS website. Peer referral was allowed to continue through to the end of data collection. Round 1 RDS was conducted in 3 MSAs: Boston, Dallas, and New York City.

Venue-Based Sampling

Venue-based sampling (VBS) is a systematic method which recruits MSM at public locations such as bars or clubs and is the methodology used for the MSM cycle of NHBS [4, 20]. VBS involves the identification of venues, days and times where MSM are known to most likely congregate to create a sampling frame. Using this frame, venues and day-time periods are randomly selected to create a sampling calendar. Systematic selection of persons from the venues takes place during scheduled days and times.

WHBS venues were not physical locations as in NHBS, but internet-based “venues” such as web sites or chat rooms. Internet venues that were thought to be popular among local MSM were identified during the formative research and through internet searches. Examples of internet venues included online chat rooms where men log in and use instant messages to have live conversations with other men, or websites where individuals post personal profiles in an attempt to meet other men. Identification of venues was not limited to those where MSM seek companionship or sex. Similar to other VBS methods, WHBS staff maximized recruitment for each identified venue by identifying 4-hour blocks of time called venue-day-time periods (VDTs) during which at least 7 MSM were expected to be enrolled. Once a list of internet venues and their respective VDTs had been assembled, staff randomly selected the VDTs to develop a weekly sampling schedule noting the days and times during which study recruitment would occur.

Recruitment methods varied depending on the type of venue where recruitment occurred: chat rooms or profile websites. Recruitment within internet chat rooms allowed live conversation via instant message between men entering the website and staff recruiters. The recruiters used private instant messages to systematically approach the next person entering the chat room when the staff member was ready to interact with another potential participant. For persons responding to the instant message, staff briefly described the study and asked a set of screening questions to determine the person’s eligibility to participate in the study. Eligible persons who agreed to take the survey received a unique study ID and a link to the secure WHBS website. Eligibility screening was repeated on entry to the WHBS website (i.e., persons were screened twice).

Recruitment within profile web sites did not always allow for instant live conversation with men, but, rather, used email links provided within each user’s profile. To recruit persons from these sites, staff systematically emailed every n^th (e.g., 3^rd, 4^th, or 5^th) personal ad posted on the site during a 24-hour block of time. The order in which postings occurred and were sampled was based on time of posting, the screen name, or some other systematic method based on the functionality afforded by the site. Emails included information about the study and a request to contact the study office via email or phone. Persons who responded to the staff recruiter’s email were screened. Those who were eligible received a unique study ID number and the link to the WHBS web site. Eligibility screening was repeated on entry to the WHBS website. Round 1 VBS was conducted in 3 MSAs: Baltimore, Los Angeles, and San Francisco.

WHBS Pilot Round 2

Sampling Overview

Upon the completion of Round 1 data collection, all local WHBS principal investigators and CDC reviewed experiences with the different methods (see Table 11) and materials and decided to implement Round 2 data collection using one consistent sampling method for all 6 MSAs. DM was deemed to excel at reaching the target sample size for a minimum of staff effort, but there remained concerns about its representativeness. RDS was deemed to have not been successful for recruitment, but may have been difficult to implement in Round 1 because of difficulty with seed selection and the requirement that previous participants generate their own emails or other messages for peer recruitment. VBS was deemed to be very labor-intensive with a relatively poor response rate. Based on these factors, the method selected was a combination of DM and RDS. Data collection for WHBS Round 2 occurred during a 16-week period in 2007; each WHBS MSA had a sample size goal of 500 MSM with complete surveys. Eligibility criteria for Round 2 were identical to those used in Round 1.

Direct-Marketing Sampling

For Round 2, all WHBS MSAs used an identical set of banner ads and fewer websites for the DM sampling. Otherwise, DM sampling methods did not differ from Round 1.

Respondent-Drive Sampling

Staff did not recruit RDS seeds in Round 2. All participants who were recruited through DM and completed the survey were considered seeds. Upon completion of the survey, DM participants were given the option of recruiting up to 3 of their peers by providing their peers’ email addresses. Email addresses were not saved after the WHBS message containing the survey link was sent. The recruiter was given an option to include a personal note to their recruits in the email. Emails contained a brief introduction to the study and a link to the WHBS website. As in Round 1, once an RDS recruit was determined to be eligible and completed the survey they too were given the option to recruit up to 3 peers.

WHBS Web Site and Survey (Round 1 and Round 2)

The secure WHBS web site could only be accessed by a unique study ID, clicking a banner ad, or using a hyperlink from an RDS email. Study IDs and hyperlinks expired if unused for 30 days. After entering the website, brief study information was provided and screening questions were administered to determine eligibility to participate. If eligible, participants were given more detailed information about the study and were invited to participate and provide consent. If they consented, the survey began immediately.

The WHBS survey consisted of seven main sections: 1) demographics; 2) MSM venue attendance and internet-usage behaviors; 3) sexual behaviors; 4) drug-use behaviors, 5) HIV and STD testing experiences; 6) accessing internet-based prevention services or information; and 7) accessing other HIV prevention services. The survey was designed to take approximately 10 to 15 minutes to complete. A participant who exited the survey early for any reason and later re-entered the WHBS site was required to restart the survey from the beginning. His previous survey responses could not be viewed by him and would appear as an incomplete survey record in the database. Once started, a participant had 24 hours to complete a survey before the link or study ID became inactive. The WHBS website did not collect any identifying information on the participants, including IP addresses, email addresses, or web logs.

Measures and Analyses

Recruitment statistics were analyzed for Round 1 (2006) and Round 2 (2007) separately for each sampling method. The count of recruitment attempts varied by method and included the number of persons that started screener questions (DM and 2006 RDS [except seeds]), the number persons that staff attempted to actively recruit (2006 RDS seeds and VBS), or the number of email recruitments sent (2007 RDS). The count of persons screened also varied by method and included the number of persons that started screener questions on the WHBS website (DM and RDS [except 2006 seeds]) or the number of persons that staff screened during active recruitment (2006 RDS seeds and VBS). The count of persons eligible only used data from the WHBS website screener questions. It was not possible to consistently count eligible persons from staff-collected VBS and RDS seed recruitment information. Some who were initially screened and were eligible during staff-led recruitment may not have accessed the WHBS website to complete the process (and would not be counted as eligible). Counts of reasons for ineligibility (not 18 years or older, not male, not an MSA resident, not RDS eligible) are also provided for the 2007 DM; specific reasons for ineligibility were not consistently tabulated in 2006. The count of persons enrolled was the same for all sampling methods and was defined as those starting the WHBS survey. The completion count was calculated based upon those completing all 3 of the final questions in the WHBS survey. Surveys with duplicate study IDs that were incomplete were excluded from the calculations for the completion rate. Since the WHBS sampling goal was based on complete surveys among MSM, the counts of men reporting sex with at least one man in the past 12 months (MSM in the past 12 months) were obtained from answers to the sexual behavior questions within the WBHS survey. For the 2007 DM method, a count of the number of banner ad impressions was available from most (but not all) websites. An impression is a measure of each time a banner ad appears on an accessed webpage and can be thought of as a passive recruitment attempt.

Demographic characteristics of participants were analyzed for 2006 and 2007 separately for each sampling method. Race/ethnicity, age category, college education, and sexual identity were selected for this report. Self-reported HIV serostatus was defined using the results of the most recent HIV test and categorized as positive, negative, or untested/unknown status. The demographics and HIV serostatus analysis was restricted to only complete surveys from those men who reported sex with a man in the past 12 months. These analysis criteria were employed to be consistent with the stated sampling goal for the pilot project.

The purpose of this research project was not to determine whether there were statistically significant differences between sampling methods. Rather, the goal was to develop an understanding of the feasibility of conducting different sampling methods and each method’s yield of completed surveys for the population of interest, MSM. Therefore no tests of significance were performed; only descriptive data are presented in this report.

RESULTS

Recruitment and Survey Completion

The DM sampling method in both rounds produced the largest numbers of complete MSM surveys and resulted in acquisition of the target sample size of 500 per MSA in 2006 and 2007 (Table 11); neither the RDS nor VBS sampling method were able to attain this goal. Though RDS had the lowest number of completed surveys of the 3 sampling methods, it also exhibited the highest eligibility rates in both rounds. In 2007, 68% (n=30,553) of 44,801 persons recruited and screened through the DM sampling method were not residents of one of the 6 WHBS MSAs; less than 1% were ineligible because they were <18 years (n=280) or not male (n=193). Enrollment rates ranged from 70% (2005 DM) to 90% (2006 VBS). Completion rates ranged from 67% (2007 DM) to 95% (2006 RDS). All methods in both rounds produced proportions of MSM in their completed surveys of greater than 80%. Few RDS participants recruited others: 8 participants in 2006 and 6 participants in 2007.

For the DM sampling method in 2007, over 21.5 million impressions resulted in 44,801 persons being screened in a 16-week period (Table 22). Persons recruited through banner ads on MySpace and Manhunt.net accounted for more than 50% of the total completed surveys, but Manhunt.net had fewer impressions and higher screening and eligibility rates than MySpace. A website specifically targeted to African American MSM, BGCOnline.com (formerly BlackGay Chat), was also one of the most common sources for WHBS participants enrolled through the DM method.

Table 2.

Recruitment Rates by Website Used for the Web-Based HIV Behavioral Surveillance System, 2007

Participant Characteristics

Overall, the MSM participants in WHBS were mostly non-Hispanic whites, between the ages of 21 and 40 years, college educated, self-identifying as homosexual and HIV-negative (Tables 3a3a and 3b3b). In 2006, the largest proportions of racial/ethnic minorities (non-white) were recruited through the RDS and VBS methods, but the 2007 DM method recruited the largest proportion of these participants. The 2007 DM method also recruited the largest proportion of young participants and those without a college education. Self-reported HIV infection ranged from 4% (2006 RDS) to 14% (2007 RDS).

Table 3a.

Demographic Characteristics of MSM Participants in the Web-Based HIV Behavioral Surveillance System, 2006

Table 3b.

Demographic Characteristics of MSM Participants in the Web-Based HIV Behavioral Surveillance System, 2007

DISCUSSION

The use of banner ads through the DM method was highly successful during both rounds of WHBS data collection. The systematic sampling methods used in the WHBS pilot were less successful but produced important insights regarding the feasibility of these methods. We found that a VBS method designed to approximate the methods used for real-world venues by NHBS was theoretically feasible using internet “venues”, but was labor-intensive, had difficulty meeting adequate enrollment proportions, and did not capitalize on some of the benefits of an internet-based sampling method (e.g., enrollment required a staff recruiter and was not automated). VBS is particularly complicated due to rapid changes in internet technology and the fluctuation of sites most commonly used by MSM for networking and socializing. RDS is another potentially viable systematic sampling option for the internet, but in both rounds of WHBS data collection, we were unable to sustain the necessary peer recruitment. RDS may not have been successful for WHBS because of the lack of incentives or the potentially weak ties between individuals of internet-based social networks; dual incentives and a strongly connected peer network have been recognized as potential preconditions for RDS success [18, 21-23]. Another internet-based research survey did successfully use RDS to recruit a sample of college students, but this success may have been supported by real-world connections between peers in that study [23].

DM resulted in a large sample of eligible MSM enrolled in a very brief period of time. These results are consistent with the experiences of several other research studies of MSM on the internet that used banner ad recruitment [7,13-16,24]. With the exception of residency-based criteria, the eligibility, enrollment, and completion rates for DM were somewhat comparable to the other WHBS methods and the few other internet-based MSM behavioral studies that have reported recruitment information [5,24-26]. Exclusive of the large number of persons ineligible because of residency, the eligibility rate, enrolment rate, and proportion who reported male-male sex in the past 12 months are consistent with the 2003-2005 MSM cycle of NHBS [3], though the survey completion rate was substantially lower for the 2007 DM method of WHBS (67%) than for NHBS (93%). The overall average click-through rate (those screened among all impressions) for WHBS banner ads was equivalent to the industry average of 0.2%, but ads on MSM-specific websites, such as Manhunt.net and BGCOnline.com (formerly BlackGayChat) did substantially better [27]. The refined DM method used in 2007 was also much more successful than the 2006 DM method at reaching racial/ethnic minority MSM and young MSM, both groups of importance to the MSM HIV epidemic and HIV behavioral surveillance.

Several limitations for the WHBS pilot study should be noted. We were unable to provide participant incentives in either round of data collection; participant incentives may be an integral part of the success of some systematic sampling methods (RDS in particular) and may have adversely impacted the success of these methods. We are unable to determine sampling bias or external validity for any of the WHBS samples. Because of the variability in the way that recruitment information was collected through different sampling methods, some of the differences in screening, eligibility, and enrollment rates may be due to the way information was collected and not to true differences between sampling methods. Because of the anonymous nature of the survey, we are also unable to determine whether an individual participated in the survey multiple times which may have been more feasible with the DM method, but we believe this would be minimized given the lack of incentives for survey participation. We also cannot determine the veracity of responses to survey questions, though social desirability bias is likely minimized by the anonymous nature of the survey. Finally, the sample sizes for VBS and RDS were inadequate for us to conduct statistical comparisons between the samples of MSM with an appropriate degree of scientific rigor, but this was also not the intent of this analysis, which was to determine the feasibility of the methods to reach target sample sizes.

The way in which MSM interact and seek sexual partnerships has expanded to include the internet, and as such, an internet-based sampling method for MSM has a role in monitoring HIV-related risk behavior. The WHBS pilot project has shown that an internet-based systematic sampling method of MSM is complex and may not be feasible, but obtaining a convenience sample of MSM using banner ads is feasible and may produce useful and timely behavioral information from a large number of MSM. Another paper published in this journal issue examines whether MSM surveyed through this sampling method differ from NHBS and explores whether internet-based sampling methods may have a role in providing behavioral surveillance information that is comparable or complementary to NHBS [17]. Any future plans for internet-based sampling of MSM will need to account for the current state of internet technology and social networking to best capitalize on the internet as an HIV research, surveillance, and prevention tool.

Human Participant Protection

The WHBS pilot project was research conducted in compliance with federal regulations governing protection of human subjects and was reviewed and approved by all relevant human subjects review boards.

ACKNOWLEDGEMENT

Declared none.

WEB-BASED HIV BEHAVIORAL SURVEILLANCE STUDY GROUP

Colin Flynn and Frangiscos Sifakis, Baltimore, MD; Eric Rubinstein and Gabriella Garcia, Boston, MA; Richard Yeager and Douglas Shehan, Dallas, TX; Trista Bingham and Alexander Carruth, Los Angeles, CA; Christopher Murrill and Sabina Hirshfield, New York City, NY; H. Fisher Raymond and Greg Rebchook, San Francisco, CA; Matthew Amsden and Tom Coggia, Reveal Communications; Paul Young and Benjamin Neal, NOVA Research Company.

CONFLICTS OF INTEREST

The authors have no institutional or commercial affiliations that might pose a conflict of interest regarding the publication of this manuscript.

REFERENCES

1. Centers for Disease Control and Prevention. HIV Prevention Strategic Plan Through 2010. Atlanta, Georgia: US Department of Health and Human Services, CDC; 2001. Available at: http://www.cdc.gov/hiv/resources/reports/psp/ [Accessed: August 31, 2011].

2. Gallagher KM, Sullivan PS, Lansky A, Onorato IM. Behavioral surveillance among people at risk for HIV infection in the U.S.: the National HIV Behavioral Surveillance System. Public Health Rep. 2007;122(Suppl 1):32–8. [PMC free article] [PubMed]

3. Centers for Disease Control and Prevention. Human immunodeficiency virus (HIV) risk, prevention, and testing behaviors--United States, National HIV Behavioral Surveillance System: men who have sex with men, November 2003-April 2005. MMWR Surveill Summ. 2006;55(6):1–16. [PubMed]

4. MacKellar DA, Gallagher KM, Finlayson T, Sanchez T, Lansky A, Sullivan PS. Surveillance of HIV risk and prevention behaviors of men who have sex with men--a national application of venue-based, time-space sampling. Public Health Rep. 2007;122(Suppl 1):39–47. [PMC free article] [PubMed]

5. Ross MW, Tikkanen R, Mansson SA. Differences between Internet samples and conventional samples of men who have sex with men: implications for research and HIV interventions. Soc Sci Med. 2000;51(5):749–58. [PubMed]

6. Benotsch EG, Kalichman S, Cage M. Men who have met sex partners via the Internet: prevalence, predictors, and implications for HIV prevention. Arch Sex Behav. 2002;31(2):177–83. [PubMed]

7. Hirshfield S, Remien RH, Humberstone M, Walavalkar I, Chiasson MA. Substance use and high-risk sex among men who have sex with men: a national online study in the USA. AIDS Care. 2004;16(8):1036–47. [PubMed]

8. Liau A, Millett G, Marks G. Meta-analytic examination of online sex-seeking and sexual risk behavior among men who have sex with men. Sex Transm Dis. 2006;33(9):576–84. [PubMed]

9. Klausner JD, Wolf W, Fischer-Ponce L, Zolt I, Katz MH. Tracing a syphilis outbreak through cyberspace. JAMA. 2000;284(4):447–9. [PubMed]

10. Tashima KT, Alt EN, Harwell JI, Fiebich-Perez DK, Flanigan TP. Internet sex-seeking leads to acute HIV infection: a report of two cases. Int J STD AIDS. 2003;14(4):285–6. [PubMed]

11. Centers for Disease Control and Prevention. Internet use and early syphilis infection among men who have sex with men--San Francisco, California, 1999-2003. MMWR Morb Mortal Wkly Rep. 2003;52(50):1229–32. [PubMed]

12. McFarlane M, Bull SS, Rietmeijer CA. The Internet as a newly emerging risk environment for sexually transmitted diseases. JAMA. 2000;284(4):443–6. [PubMed]

13. Chiasson MA, Hirshfield S, Remien RH, Humberstone M, Wong T, Wolitski RJ. A comparison of on-line and off-line sexual risk in men who have sex with men: an event-based on-line survey. J Acquir Immune Defic Syndr. 2007;44(2):235–43. [PubMed]

14. Hidaka Y, Ichikawa S, Koyano J, et al. Substance use and sexual behaviours of Japanese men who have sex with men: a nationwide internet survey conducted in Japan. BMC Public Health. 2006;6:239. [PMC free article] [PubMed]

15. Elford J, Bolding G, Davis M, Sherr L, Hart G. Web-based behavioral surveillance among men who have sex with men: a comparison of online and offline samples in London, UK. J Acquir Immune Defic Syndr. 2004;35(4):421–6. [PubMed]

16. Carballo-Dieguez A, Miner M, Dolezal C, Rosser BR, Jacoby S. Sexual negotiation, HIV-status disclosure, and sexual risk behavior among Latino men who use the internet to seek sex with other men. Arch Sex Behav. 2006;35(4):473–81. [PubMed]

17. Sanchez T, Smith A, Denson S, DiNenno E, Lansky A. Internet-based methods may reach higher-risk men who have sex with men not reached through venue-based sampling. Open AIDS J. 2012;6(Suppl 1):83–9. [PMC free article] [PubMed]

18. Magnani R, Sabin K, Saidel T, Heckathorn D. Review of sampling hard-to-reach and hidden populations for HIV surveillance. AIDS. 2005;19(Suppl 2):S67–72. [PubMed]

19. Lansky A, Abdul-Quader AS, Cribbin M, et al. Developing an HIV behavioral surveillance system for injecting drug users: the National HIV behavioral surveillance system. Public Health Rep. 2007;122(Suppl 1):48–55. [PMC free article] [PubMed]

20. MacKellar D, Valleroy L, Karon J, Lemp G, Janssen R. The young men's survey: methods for estimating HIV seroprevalence and risk factors among young men who have sex with men. Public Health Rep. 1996;111(Suppl 1):138–44. [PMC free article] [PubMed]

21. Abdul-Quader AS, Heckathorn DD, Sabin K, Saidel T. Implementation and analysis of respondent driven sampling: lessons learned from the field. J Urban Health. 2006;83(Suppl 6):i1–5. [PMC free article] [PubMed]

22. Ramirez-Valles J, Heckathorn DD, Vazquez R, Diaz RM, Campbell RT. From networks to populations: the development and application of respondent-driven sampling among IDUs and Latino gay men. AIDS Behav. 2005;9(4):387–402. [PubMed]

23. Wejnert C. An empirical test of respondent-driven sampling: point estimates, variance, degree measure, and out-of-equilibrium data. Sociol Methodol. 2009;39(1):73–116. [PMC free article] [PubMed]

24. Rosser BR, Miner MH, Bockting WO, et al. HIV risk and the internet: results of the men's interrnet sex (MINTS) study. AIDS Behav. 2009;13(4):746–56. [PMC free article] [PubMed]

25. Ross MW, Tikkanen R, Mansson SA. Differences between internet samples and conventional samples of men who have sex with men: implications for research and HIV interventions. Soc Sci Med. 2000;51(5):749–58. [PubMed]

26. Ross MW, Rosser BR, Stanton J, Konstan J. Characteristics of Latino men who have sex with men on the internet who complete and drop out of an internet-based sexual behavior survey. AIDS Educ Prev. 2004;16(6):526–37. [PubMed]

27. eMarketer. The Latest Ad Click Count. Available at: http://www.emarketer.com/Articles/Print.aspx?1006969 . 2011. [Accessed August 31, 2011].

Articles from The Open AIDS Journal are provided here courtesy of
Bentham Science Publishers