The quality of sexual-behavior assessment is an important but daunting challenge to HIV-related research. From the research produced to date, there is evidence suggesting that well-designed interviews and questionnaires can provide acceptable data when administered appropriately. However, the problematic issues discussed herein (see also
Catania et al., 1990,
1995), coupled with the public health implications of the use of sexual self-report data, require active research that focuses on the design and evaluation of measures for a variety of populations. To improve the quality of self-report assessment of sexual behavior, we offer the following recommendations, which are based on the reviewed studies, information from literacy experts, and our experiences with survey design and administration. These recommendations are likely to be useful for selecting among existing measures, adapting existing measures for different applications, or designing new measures.
Use psychometrically evaluated measures
provides psychometric evidence for the latest measures developed. Research on the Risk Behavior Assessment (RBA;
Needle et al., 1995), which includes sexual and drug-related HIV risk factors, provides an example of how psychometric evaluation can be conducted systematically. Psychometric properties of the RBA measure have been evaluated with different populations. When adapting a measure for use in a population, or when creating a new measure, it is crucial to conduct appropriate psychometric evaluation. In studies reporting test-retest analyses, the focus should be on the consistency of reporting rather than the consistency of target behaviors. Consequently, investigators should be careful to assess behaviors occurring in the same reporting period at both administrations. As others have pointed out, shorter assessment intervals (e.g., one week) limit the impact that memory deterioration and other variables may have on retest coefficients.
Validate the measure
There is an urgent need for a standardized method for validating findings from sexual behavior self-reports. One approach, suggested by Schopper et al., (1993), is to (i) use the concordance of numbers of regular and casual partners and sex acts across gender to identify gaps; (ii) use the degree of concordance between couples on the type of relationship (e.g., monogamous or polygamous), and on number of sex acts with the partner to measure accuracy at the individual level; and (iii) compare patterns related to participant demographics and behavior with results from other surveys with similar samples.
Seidman and Reider (1994) reviewed sexual behavior surveys conducted in the United States, and provided data on “normative” sexual behavior across the life cycle. A more comprehensive survey has recently been reported by
Laumann et al. (1994). This normative data may allow researchers to compare aggregate patterns of behavior in their samples with appropriate population parameters. Other methods can be used to provide evidence for validity of self-report measures are biochemical evidence of sexual activity (e.g., reinfection with STDs) and comparison with concurrent self-monitoring data.
Use a measure appropriate for the purpose of the assessment
Because measurement instruments address different levels of behavioral specificity, attention to the needs of the assessment procedure are warranted. It may be useful to conceptualize measures of HIV-risk behavior as falling into to three concentric categories: risk screening, risk assessment, and risk-event data.
Risk screening describes dichotomously whether a respondent has engaged in risk behavior during the reporting period (e.g.,
Boekeloo et al., 1994).
Risk assessment allows description of the level of risk, based on the frequency of risk behavior (e.g.,
Downey et al., 1995).
Risk-event data is the most detailed, and allows event-level examination of the co-occurrence of potential risk factors with risk behaviors (e.g.,
Crosby et al., 1996). Clearly, a measure designed for risk-screening may yield data that are inadequate if the goal of an investigation is to assess the level of risk reported by the target sample. Alternatively, although an event-level assessment provides risk screening data, the additional resources required for assessment would be wasted if event-level analysis is not required.
Use language that is easily understood
Researchers interested in studying at-risk behaviors among under-served samples are likely to encounter problems associated with the terminology used in assessments. On the one hand, terms that are unfamiliar to participants may lead to increased measurement error, and on the other hand, using slang terms from the argot of the target sample may reduce researchers’ credibility or be misinterpreted as condescension. One solution is to provide parenthetical descriptions of clinical terms. For example, “How many times in the past three months have you had vaginal sex (when a man puts his penis in a woman’s vagina)?” Careful pilot research with the target sample prior to measurement development and implementation is essential. Specifically, the literacy level of the population should be assessed, and used to guide vocabulary used in the questionnaire. Oral and written instructions should be as concise as possible, and should be reviewed with participants before they begin the questionnaire If multiple questionnaires are used, response formats should be as consistent as possible. Finally, offering an audiotape-administered version may be advantageous for groups with low reading levels.
Use focus groups, pilot data, and other formative methods to adapt the assessment protocol for sensitivity to cultural issues of the participants
Just as consideration of the reading and comprehension level of the target sample is necessary, so to is attention to the lifestyles, prevailing customs, and religious and cultural traditions of target samples. For example, sexual behavior questionnaires should be designed without inherent sexual orientation bias. Caution should be exercised to include response options that are suitable for a wide variety of responses. Filler questions should be culturally relevant. Also, emphasizing the personal benefits of the research to participants and their community is likely to engender cooperation and elicit candid reporting. Conducting focus groups with men and women from the population to be assessed often reveals unanticipated cultural and contextual issues that can be used to inform sexual behavior assessment (
Carey et al., 1997). The need for qualitative research during measurement development cannot be over-emphasized.
Include techniques that improve recall of behavior
Croyle and Loftus (1993) detailed the influence of the constructive nature of memory on self-reports of sexual behavior. Simple forgetting, telescoping (distorting the recency of particularly memorable events), exposure to misleading information since the event, and the use of heuristics to estimate behavior frequencies are some of the factors that can contribute to inaccurate self-reports. Useful techniques include: (a) providing anchor dates for reporting periods, (b) encouraging participants to use appointment books and calendars to recall other memorable events during the reporting period, and (c) recalling extensive periods of abstinence or consistent sexual activities. Recent use of the timeline follow-back procedure (
Sobell & Sobell, 1996), which utilizes many of these techniques, to assess sexual behavior have shown that it provides valuable event-level data (
Crosby et al., 1996) and that these self-reports are reliable (
Weinhardt et al., in press).
Establish a working trust with interviewees and questionnaire respondents
Assessment of risk should take place after a participant and interviewer have established rapport, and the interviewer has assured the participant of confidentiality. Specific sexual behavior assessment should always begin with an appropriate introduction for the participants (
Carey, 1998). During this time the reasons for asking questions about sexual (and other socially sensitive) behaviors should be provided. For example, in a clinical context, one might say that a standard practice is to inquire about risk for HIV just as one routinely inquires about suicidal ideation, personal safety, and other important matters; thus, all participants get asked and no one will feel singled out as being at unique risk. In a public health or research context, investigators might identify the overall purpose of the research, including how this will improve public health or enhance the scientific knowledge base. After these introductory remarks, participants should be invited to ask any questions they might have.
If an interviewer or investigator appears embarrassed about or unsure of the appropriateness of the questions, participants may detect this and provide incomplete or ambiguous responses.
Adopt default assumptions to gather the most accurate information efficiently (
Wincze & Carey, 1991)
These assumptions reflect the preferred direction of error. Thus, for example, it is better to assume minimal understanding on the part of the participant so that language is clear and concrete. Other useful assumptions include: (i) participants will be embarrassed about and have difficulty discussing sexual matters; (ii) participants will not understand all sexual behavior terms, medical terminology, etc.; (iii) participants will be misinformed about sexual material, including STDs and other threats to sexual health; and (iv) questions will provoke concern regarding a participant’s health or well-being (thus, the interviewer should be prepared with information, referral sources, and other reassuring materials). As the interviewer or investigator learns more about the client, these assumptions can be adjusted.
Sequence the inquiry from the least to most threatening questions
Thus, questions about oral or vaginal sex might precede questions regarding sexual behaviors that are less socially approved (e.g., anal intercourse, fisting, or rimming).
Rather than ask “if” a clients has engaged in a particular activity, ask “how many times have you ...” engaged in it. Use of open response formats on questionnaires is encouraged (
Catania et al., 1990). Such an approach will communicate an investigator’s (or clinician’s) expectation that such behaviors do occur and are not abnormal.
Be sensitive to contextual issues in administration
Among these factors are the experimenter’s demeanor during interactions, administration setting, relevance of the studies aims to participants’ lives, and perceptions of trust regarding the purpose and personnel of the study. Clearly, these “non-specific” variables may impact participants’ the assessment protocol and affect measurement error.
Additional studies employing these steps will increase understanding of the issues affecting sexual behavior self-reports, and facilitate the development of standardized self-report measures that are appropriate for specific purposes in a variety of populations. Researchers and practitioners can then place more confidence in sexual behavior data gathered with self-report methods.