PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Adolesc Health. Author manuscript; available in PMC 2010 August 1.
Published in final edited form as:
PMCID: PMC2752864
NIHMSID: NIHMS134298

What are the Consequences of Relying upon Self-Reports of Sexually Transmitted Diseases? Lessons Learned about Recanting in a Longitudinal Study

Abstract

Self-reports are the standard measure of STD history employed in survey research. To what extent are self-reports of ever having an STD recanted in a follow-up data collection? Using the National Survey of Adolescent Males (NSAM), we assess consistency over time in self-reports of ever having an STD in a sample of young men transitioning from adolescence to young adulthood (age 15-26), a population in which STDs are particularly prevalent. Approximately 7% of all sexually experienced young men rescinded STD self-reports over time. Thus, self-reports at one point in time likely underestimate true STD history, using earlier self-reports as the criterion. Among men who ever report an STD, 94-98% recant their reports in later waves. Knowledge of the extent of underreporting can potentially be used to adjust cross-sectional estimates of STDs based on survey self-reports. This study moves us one step closer to estimating just how much underreporting of STDs in self-reports is.

Keywords: Adolescent Males, Recanting, Sexually Transmitted Diseases, Self-Reports

INTRODUCTION

Why Should We Care about STD Self-Report Integrity?

Many national health surveys including the National Survey of Family Growth (NSFG), the National Health Interview Study (NHIS, 1994 - Year 2000 Objectives Supplement), the Behavioral Risk Factor Surveillance System (BRFSS), the National Longitudinal Survey of Adolescent Health (ADD-Health waves 1 and 2), and the National Health and Social Life Survey (NHSLS) ask respondents to report whether or not they have had a sexually transmitted disease (STD). Some do this with a global question about having had any STD, or any of a list of specific STDs (e.g., NSFG, NHIS, NHSLS). Other surveys collect self-reports about individual diseases such as herpes, chlamydia, gonorrhea, syphilis, and HIV/AIDS (e.g., NSFG - follow-up items, BRFSS, ADD-Health). It is generally accepted that respondents’ self-reports of conditions and behaviors can be affected by reporting error such as memory failure, misunderstanding of the question, or social desirability, among other factors. O’Sullivan [1] said it succinctly “somewhat surprisingly at times, we continue to conduct research as if the factors invalidating our work are nonexistent or reducible to statistical error” (p. 207). In this paper we examine the extent to which positive self-reports of STDs are rescinded over time and point out the questionable validity of these measurements that are extensively and commonly used.

In most cross-sectional surveys there is no practical way of assessing the potential discrepancy between the self-report and the individual’s true status. If the self-report concerns current status, its accuracy can be assessed by comparison to clinical measures, but this is costly and therefore rarely done. In addition, if the self-report concerns whether the respondent has ever had an STD, clinical measures cannot necessarily reveal past instances of STD infections.

In longitudinal surveys, asking the same questions across time may provide insight into the accuracy of self-report measures, through analysis of the consistency of reports across time. In this article we use this approach with the National Survey of Adolescent Males (NSAM) to explore the level of error in young men’s reports about ever having an STD, specifically focusing on one aspect1 of self-report validity: whether positive self-reports are later rescinded. Although there are multiple ways to assess the accuracy of self reports, analysis of recanting has been widely used in other areas: substance use [2-10], sexual behavior [6, 11, 12], virginity pledges [13], and virginity [14]. No one method is perfect, but each contributes to our understanding of the validity of self-reports, making it possible to triangulate across methods. A particular way that recanting analysis complements the use of other methods (e.g., clinic-based) is that this method can be used with national sample data and applied to interpreting findings based on self-report data.

STD incidence and prevalence among youth

The incidence of STDs among youth is a critical public health concern in the United States. Youth are dramatically disproportionately contributing to new cases of STDs. Youth aged 15 to 24 contribute to half of new STD infections each year despite constituting only one-quarter of the sexually experienced population aged 15-44 [15]. With respect to specific STDs, youths’ contribution is even higher: nearly 75% of chlamydia cases and 60% of gonorrhea cases [16]. In 2005, the percent of chlamydia diagnoses for females and males attending STD clinics increased to 15.4 and 20.5 respectively [17]. These rates are likely to be misleading of actual rates given that women and men attending clinics are a unique group of people motivated to receive services for various reasons (e.g., presenting symptoms, heightened health awareness, pregnancy). Asymptomatic individuals and those not currently experiencing a pregnancy most likely do not seek clinical services.

Assessing the survey standard

Self-reports serve as the standard measure of STD history, sexual experience, and substance use in survey research. Due to the sensitive nature of these questions and in order to reduce potential social desirability bias, most surveys use self-administered techniques (i.e., self-administered questionnaire - SAQ; audio-computer assisted self-interviewing - ACASI) to collect these data.

We define recanting as admitting to some lifetime (“ever”) behavior at one time followed by denial of the lifetime behavior at a subsequent time. Rescinding responses makes suspect the self-report behavioral measures used in survey research. To our knowledge, recanting of elf-reported STDs among youth has not been reported. Recanting of other self-reported risk behaviors has been studied more extensively, especially substance use [2-10, 18-20] and sexual behavior [11-14, 19-21]. Research on inconsistencies or “rescinding” of lifetime reports of sexual experience indicates that the time interval between measures and the sex and race of youth predicts the extent of retracting sexual experience status.

Orr and Fortenberry [21] found that among 15-19 year old women, 7% claimed virgin status only three months after admitting to being sexually experienced. Upchurch and colleagues [12] found over 11% of youth who reported sexual experience at the first wave of a longitudinal study denied being sexually active at a later wave (1 year later) and 78% of youth who consistently reported sexual experience over time revised the date of sexual initiation by 2.6 to seven months later than originally reported. For both lifetime recanting and sexual debut date revision, males reported higher rates [12].

Rosenbaum [13] discovered that over half of youth who report taking a virginity pledge at an earlier time denied they ever made such a pledge one year later. And, youth who rescinded virginity pledges were four times as likely to rescind sexual experience. (they became “born again virgins”). Additionally, Palen and colleagues [14] found that among South African youth who reported being sexually experienced at the beginning of eighth grade, nearly 40% recanted by claiming to be virgins at the beginning of the tenth grade. Youth who rescinded sexual experience were also more likely to retract reports of substance use over time relative to youth who did not rescind sexual experience.

Focusing on STDs among African American adolescents (predominantly females) in a clinic setting, Clark and colleagues [22] used structured interviews to assess the degree of accuracy between youth self-reports of visits for STDs and actual visits noted in their medical charts. After six to 12 months, 40% reported no STD visits when their charts revealed at least one visit. Clark and colleagues speculate that these inconsistencies may be due to the face-to-face nature of structured interviews which yield fewer reports of sensitive behaviors. But, these results are consistent with the recants Palen et al. [14] found for sexual experience using self-administered techniques.

Using a design similar to the one used by Clark et al, a second study examining STDs, conducted by Harrington and colleagues [23], used clinical STD test results at baseline and follow-up self-report surveys one month later to assess the validity of STD reports. Among a sample of 522 African American women ages 14 to 18, they documented that of the women who tested positive at baseline and were notified they had an STD, only 69% self-reported ever having an STD one month later.

Current study

In a recent editorial, O’Sullivan [1] highlights the importance of studying self-reported data and calls for a “renewed commitment” to examining the reliability and validity of survey measures and their underlying constructs. This study answers this call. Although there are multiple studies of reporting accuracy concerning other sex-related topics, with the Clark et al. and Harrington et al. studies, there are only two prior studies specifically concerning accuracy of STD reports, both using clinic-based samples, one exclusively focusing on females and the other with an overrepresentation of females in the sample. Our research adds a third study to this limited literature that (1) focuses on men, (2) uses data from a national representative sample rather than a clinic sample, and (3) longitudinally explores changes and stability in self-reported STDs over seven years of development (from adolescence into early adulthood).

In this paper we use data from a longitudinal study of a nationally representative sample of men, NSAM, to assess the degree to which men inconsistently self-report lifetime STD status over two or three times of measurement. A literature search identified no prior studies that have examined the consistency of longitudinal STD self-report measures among adolescent and/or young adult men. Our results alert us to the underestimation of STD history when relying upon the STD self-report items employed in survey research.

METHOD

Data

Data used in this study were obtained in three waves as part of the National Survey of Adolescent Males (NSAM). The first wave of data was collected from a nationally representative sample of non-institutionalized, unmarried men ages 15 to 19 in 1988 (wave 1). These men were followed up at ages 17 to 22 in 1990-1 (wave 2) and at ages 21 to 26 in 1995 (wave 3). African American and Hispanic men were oversampled in the original sample and sampling weights were created to adjust for the oversampling to make results characterize the population. During each wave of NSAM, young men were interviewed at home (1-2 hours) about their sexual and reproductive health with emphasis on attitudes, sexual behaviors, partnering, condom use, and sexually transmitted infections. The most sensitive data - including STD diagnosis - were collected via self-administered questionnaires (paper and pencil forms). Longitudinal weights developed to adjust for non-response were used [10].

Sample

These analyses are limited to 1,263 respondents who (1) participated in at least two waves, (2) ever had sex with a female in at least two waves, and (3) completed the STD items in the SAQ portion of the survey in each wave (See Appendix A for details). Given our interest in examining recanting between and among waves because timing of observations may matter, we conducted analyses in different subsamples according to waves of participation. Of 1,880 original respondents in the first wave, 1035 men met the inclusion criteria for waves 1 and 2; 791 met the criteria for waves 1 and 3 irrespective of wave 2; and 976 met the criteria for waves 2 and 3 regardless of wave 1. These three samples overlap and are not mutually exclusive. Furthermore, 957 respondents participated in and met the inclusion criteria across all three waves.

Measures

Self-reported history of STDs

Respondents were asked the following yes-no question in all three waves in an SAQ: “Have you ever been told by a doctor or nurse that you had either gonorrhea or ‘clap,’ syphilis, herpes, or Chlamydia?” This item was asked in the SAQ portion of the survey. In the last wave of the survey, some respondents could not be reached in-person, but participated via telephone. These respondents were sent the paper-and-pencil SAQ rather than administered the questions over the phone, and were asked to return their forms via mail;

Sexually experience

The denominators used in some of the results below are male respondents who reported any sexual activity with females for at least two waves (these men may additionally have reported having had sex with men). We calculated estimates of recanting STD history using (1) only sexually experienced youth, given that these are the individuals at risk for contracting an STD, and (2) only those youth who report an STD at an earlier wave.

Data Analysis: What Constitutes a Recanted Self-Report across Time?

In principle, a recanted report across waves occurs when a respondent answers “yes” in an earlier wave and “no” in a later one2. When data from two waves are considered, a rescinded report takes the form “yes” in the earlier wave followed by “no” at the later wave. With data from three observations, a recanted report takes on four possible forms: “yes” in the first wave followed by “no” in the second and third waves; “yes” in the first two waves followed by “no” in the third wave; “no” in the first and third waves with a “yes” in the second wave; and “yes” in the first and third waves with a “no” response in the second wave.

RESULTS

Results for the Total Eligible Sample

Of the total sample of 1,263 respondents with two-or-more data points, 99 (7.1 %, weighted) rescinded their self-reports of STD history across time (Table 1, column 5; Figure 1, center circle). This total sample includes men who participated in wave 1 and 2 only (13 rescinded their STD status), wave 1 and 3 only (2 rescinded), and waves 1, 2, and 3 (84 rescinded). Hereafter we will characterize the three waves as middle adolescence, late adolescence, and early adulthood, respectively.

Figure 1
Flow Chart of Rescission Between and Among Waves, Weighted Percents
Table 1
Inconsistent STD Reports - Percent and Number Recanting

Recanting From Middle to Late Adolescence

Recanting from middle to late adolescence was calculated among all men who were sexually experienced in waves 1 and 2 irrespective of participation in wave 3. Approximately 2.5 years occur between these two waves. Of the 1035 respondents meeting the inclusion criteria, 43 (3.0%, weighted) reported they ever had an STD by the first wave and reported they never had an STD by the second wave (Table 1, column 1; Figure 1).

Recanting From Late Adolescence to Early Adulthood

Recanting from late adolescence to early adulthood was calculated among all men who were sexually experienced in wave 2 and 3. The time between waves spans four years. Of the 976 men reporting being sexually experienced in waves 2 and 3 and completed STD items, 60 (5.3%, weighted) rescinded their report of ever having an STD in wave 2 by claiming they never had an STD in wave 3 (Table 1, column 3; Figure 1).

Recanting From Middle Adolescence to Early Adulthood

Recanting from middle adolescence to early adulthood was calculated among all men who were sexually experienced in waves 1 and 3. This sample has the fewest men due to attrition and to the fact that more men reported virgin status in wave 1 than any other wave. Of the 791 men who answered STD items and were sexually experienced in waves 1 and 3, 18 (1.9%, weighted) rescinded their STD status in wave 3. Despite seven years of time between these waves, recanting between these two waves is the smallest (Table 1, column 2; Figure 1). To further understand this pattern, we examined recanting patterns for men who met the inclusion criteria across all three waves.

Recanting Across Middle Adolescence, Late Adolescence, and Early Adulthood

Of the 957 respondents participating in all three waves of data collection, who completed STD items, and who reported being sexually experienced during all three waves or the last two waves only, 7.1 percent (weighted, N=84) rescinded STD reports over time (Table 1, column 4; Figure 1). The most prevalent recanting pattern was respondents who reported “no” in the first wave, “yes” in the second wave, and “no” in the last wave (N=57) followed by men who reported “yes” in the first wave and no for the remaining two waves (N=14), and men who reported “yes” in the first and last waves and “no” for the second wave (N=11). The least likely recanting pattern was for men who reported “yes” in the first two waves and “no” in the last wave (N=2). The most common pattern revealed here - recanting during the second wave - was not captured above in recanting between waves 1 and 3, in turn, helping to explain the observed dip in recanting between waves 1 and 3 not considering wave 2. In short, self-reports differentially underestimate STD status depending on when the measure is taken, a finding that cross-sectional studies cannot reveal.

Recanting Among Men Who Report Ever Having an STD

Above results pertain to the whole sample of sexually experienced youth. Now we turn to youth who report ever having an STD in wave 1 or 2. As shown in Table 2, 44 men reported having an STD in the first wave. Nearly all (98.1%, weighted) of these men later recanted his report in wave 2 or wave 3. Of the 94 men reporting ever having an STD in wave 2, 94% (weighted) rescinded their reports in wave 3.

Table 2
Recanting in Subsamples Previously Reporting an STD

DISCUSSION

Summary of Results

Of the at-risk, sexually experienced sample contributing data for three time points, 7% rescind reports of whether a doctor ever told him he had an STD. From the first to the third wave there was nearly 30% attrition; men who engage in more risk behaviors were more likely to attrit [24]. Hence, these findings may underestimate the gravity of reporting errors. Furthermore, recanting patterns reveal that retracting reports of STD history does not occur at a steady rate over time. We observed that 2% recanted between middle adolescence (wave 1) and early adulthood (wave 3), across 7 years of development; 3% of men rescinded from middle adolescence to late adolescence (wave 2); 5.3% rescinded between late adolescence to early adulthood, across 4 years of development; and 7.1% rescinded across all three developmental stages (Figure 1).

The most common recanting pattern among men participating in all three waves - initially reporting “no” in middle adolescence then reporting “yes” in late adolescence when risk is known to peak, and finally reporting “no” in early adulthood when settling down into relationships occurs - suggests that cross-sectional studies based on reports during middle adolescence and early adulthood are more vulnerable to underestimating STD status and history than when based on late adolescence. When we take into account wave 2 data, the anomalous 1.9% recanting between waves 1 and 3 changes dramatically to 7.1. More revealing is the finding that of men who report an STD in an earlier wave, 94% or more later recant their reports.

Putting These Results in Perspective

It is unlikely that most future surveys asking about STD status and history will collect biological specimens. In light of the continued use of self-report STD measures, alone, in surveys, the purpose of our analyses is to call our attention to the underreporting bias inherent in them. Collecting these self-reports in SAQ format, as in this study, does help to minimize response bias [25]. Nonetheless, 2 to 7% recanting rates among the sexually experienced sample suggest that underreporting is still occurring, and warrants our attention. Although the inconsistencies may have limited effect on estimates of prevalence, they will have much greater impact on analyses of predictors and covariates of STD’s given even small amounts of measurement error greatly reduce statistical power.

Further, in light of the differential attrition of higher-risk males, and other evidence that higher-risk males are more likely to rescind reports of substance use, our estimates of STD recanting are likely underestimates. Unfortunately we cannot assess the extent to which respondents underreport STDs at time 1 or the degree to which respondents unknowingly have an STD, both additional sources of error threatening the validity of self-reports. But, we do know that more than 9 in 10 of young men who reported an STD in an earlier wave later retracted his report of ever having an STD. Future research that validates self-reports against clinical measures will contribute further knowledge of the extent of STD underreporting. Estimates of underreporting based on recanting analysis as well as comparison of survey and clinical measures can potentially be used to adjust cross-sectional estimates of STDs based on survey self-reports. These results suggest a 7% underreporting by sexually experienced youth. Replication studies are needed to move us toward estimating just how much underreporting of STDs in self-reports occurs.

Acknowledgements

We would like to thank Janet Rosenbaum for her insightful comments and suggestions on this manuscript.

Funding

This research was supported by the National Institutes of Health - NICHD: R01 HD036948. Part of the work reported here was also supported by the Cooperative State Research, Education and Extension Service, U. S. Department of Agriculture, under Project No. ILLU-45-0366 to Joseph H. Pleck.

Appendix A

Flow Chart of Eligible Sample

An external file that holds a picture, illustration, etc.
Object name is nihms-134298-f0002.jpg

Footnotes

1We recognize that initial reports of never having an STD may also be invalid because, among other reasons, respondents may be unaware of having asymptomatic STDs or respondents may not want to answer in the affirmative due to the sensitive nature of the topic.

2We acknowledge that the assumption about the accuracy of “yes” responses in earlier waves and inaccuracy of “no” responses in later waves is a limitation of the method we are using. .

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1. O’Sullivan LF. Challenging Assumptions Regarding the Validity of Self-Report Measures: The Special Case of Sexual Behavior. Journal of Adolescent Health. 2008;42:207–208. [PubMed]
2. Fendrich M, Rosenbaum DP. Recanting of substance use reports in a longitudinal prevention study. Drug Alcohol Depend. 2003;70:241–253. [PubMed]
3. Griesler PC, Kandel DB, Schaffran C, Hu M, Davies M. Adolescents’ Inconsistency in Self-Reported Smoking A Comparison of Reports in School and in Household Settings. Public Opinion Quarterly. 2008;72(2):260–290. [PMC free article] [PubMed]
4. O’Malley PM, Bachman JG, Johnston LD. Reliability and consistency of self-reports of drug use. International Journal of Addiction. 1983;18:805–824. [PubMed]
5. Percy A, McAlister S, Higgins K, McCrystal P, Thornton M. Response consistency in young adolescents’ drug use self-reports: a recanting rate analysis. Addiction. 2005;100(2):189–196. [PubMed]
6. Flisher AJ, Evans J, Muller M, Lombard C. Test-retest reliability of self-reported adolescent risk behavior. Journal of Adolescence. 2004;27(2):207–212. [PubMed]
7. Johnson TP, Mott JA. The reliability of self-reported age of onset of tobacco, alcohol, and illicit drug use. Addiction. 2001;96:1187–1198. [PubMed]
8. Shillington AM, Clapp JD. Self-report stability of adolescent substance use. Drug Alcohol Depend. 2000;60:19–27. [PubMed]
9. Johnston LD, O’Malley PM. The recanting of earlier reported drug use by young adults. The Validity of Self-Reported Drug Use: Improving the Accuracy of Survey Estimates. NIDA Research Monographs. 1997;167:59–80. [PubMed]
10. Mensche BS, Kandel DB. Underreporting of substance use in a national longitudinal youth cohort. Public Opinion Quarterly. 1988;52:101–124.
11. Alexander CS, Somerfield MR, Ensminger ME, et al. Consistency of adolescents’ self-report of sexual behavior in a longitudinal study. J Youth Adolesc. 1993;22:455–471.
12. Upchurch DM, Lillard LA, Aneshensel CS, et al. Inconsistencies in reporting the occurrence and timing of first intercourse among adolescents. Journal of Sex Research. 2002;39:197–206. [PubMed]
13. Rosenbaum JE. Reborn a virgin: Adolescents’ retracting of virginity pledges and sexual histories. American Journal of Public Health. 2006;96:1098–1103. [PubMed]
14. Palen L, Smith EA, Caldwell LL, et al. Inconsistent reports of sexual intercourse among South African High School Students. Journal of Adolescent Health. 2008;42:221–227. [PMC free article] [PubMed]
15. Weinstock H, Berman S, Cates W., Jr. Sexually Transmitted Diseases among American Youth: Incidence and Prevalence Estimates, 2000. Perspectives on Sexual and Reproductive Health. 2004;36(1):6–10. [PubMed]
16. US Department of Health and Human Services. CDC Progress Review: Sexually Transmitted Diseases. 2004. Healthy People 2010. [Online]. Available at http://www.healthypeople.gov/data/2010prog/focus25/
17. Centers for Disease Control and Prevention . Sexually Transmitted Disease Surveillance, 2005. U.S. Department of Health and Human Services; Atlanta, GA: Nov, 2006.
18. Pleck JH, Sonenstein FL, Ku L. Black-White differences in adolescent males’ substance use: Are they explained by under-reporting by Blacks? Journal of Gender, Culture, and Health. 1996;2:247–265.
19. Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, Ross JG. Reliability of the 1999 Youth Risk Behavior Survey questionnaire. J Adolesc Health. 2002;31:336–342. [PubMed]
20. Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the Youth Risk Behavior Survey questionnaire. Am J Epidemiol. 1995;141:575–580. [PubMed]
21. Orr DP, Fortenberry JD. Validity of self-reported sexual behaviors in adolescent women using biomarker outcomes. Sexually Transmitted Diseases. 1997;24:261–266. [PubMed]
22. Clark LA, Brasseux C, Richmond D, Getson P, D’angelo LJ. Are adolescents accurate in self-report of frequencies of sexually transmitted diseases and pregnancies? Journal of Adolescent Health. 1997;21:91–96. [PubMed]
23. Harrington KF, DiClemente RJ, Wingood GM, et al. Validity of Self-reported Sexually Transmitted Diseases Among African American Female Adolescents Participating in an HIV/STD Prevention Intervention Trial. Sexually Transmitted Diseases. 2001;28:468–471. [PubMed]
24. Sonenstein FL, Ku L, Lindberg LD, Turner CF, Pleck JH. Changes in sexual behavior and condom use among teenage men: 1988 to 1995. American Journal of Public Health. 1998;88:956–959. [PubMed]
25. Turner CF, Ku L, Rogers SM, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science. 1998;280:867–873. [PubMed]