|Home | About | Journals | Submit | Contact Us | Français|
In the Fall of 2004, 1,948 5th grade students from Phoenix, AZ enrolled in an evaluation of a school-based, substance use prevention intervention. To assess the consistency of Mexican and Mexican-American students’ self-reports of lifetime substance use, the present study analyzed data reported by 1,418 students who reported Mexican ancestry and completed 2 to 6 questionnaires administered over a 40-month period. By wave 6, which was completed in March 2008, lifetime alcohol, cigarette, marijuana, and inhalant use rates were 86.0%, 65.0%, 64.5%, and 62.1%, respectively. Corresponding rescission rates were 24.0%, 9.6%, 5.8%, and 9.2%. Reporting patterns with one “Yes–No” sequence accounted for more than 88% of the inconsistent self-reports. This finding suggests that the majority of Mexican/Mexican-American preadolescents participating in a substance use prevention intervention provided logically consistent self-reports of lifetime substance use.
Longitudinal studies provide an efficient and cost-effective means of monitoring substance use among American youths and assessing the progress made toward meeting national health objectives. Additionally, longitudinal studies are ideally suited to investigate the consistency of respondents’ self-reports of substance use (see Bailey, Flewelling, & Rachal, 1992; Fendrich & Kim, 2001; Fendrich & Mackesy-Amiti, 2000; Fendrich & Rosenbaum, 2003; Fendrich & Vaughn, 1994; Johnston & O’Malley, 1997). Studies that investigate the consistency of individuals’ self-report of lifetime substance use rest on two assumptions. The first assumption is that once individuals have reported that they have used a substance, they should report that they have used the substance when they are asked the same or a logically consistent question on a subsequent questionnaire. The second assumption is that individuals use the same cognitive strategies to retrieve the same information and render the same report they gave initially. Given these assumptions, any subsequent report of lifetime use that is logically inconsistent with an earlier report always indicates response error. Either the initial report was in error (and any resulting prevalence estimate was negatively biased) or the subsequent report was in error (and any prevalence estimate was positively biased).
Table 1 summarizes key features of the studies that are frequently cited by researchers who have examined the consistency of individuals’ substance use self-reports. Mensch and Kandel (1988) conducted one of the first studies to use longitudinal survey data to study the consistency of these self-reports. Specifically, they compared marijuana use responses reported in 1980 and 1984 by individuals participating in the National Longitudinal Survey of Youths (NLSY). These survey data provided a snapshot of the self-reported substance use of a nationally representative sample of U.S. males and females who were between the ages of 14 and 21 in 1979. Mensch and Kandel found that 4% of males and 6% of females who reported marijuana use at the 1980 survey did not report use at the 1984 survey. Additionally, they found that low-level users were more likely to provide an inconsistent report than were other users.
Fendrich and Vaughn (1994) used NLSY data to investigate the consistency of lifetime marijuana and cocaine use reports given in 1984 and 1988. They found that only 19% of the individuals who reported cocaine use and only 12% of the individuals who reported marijuana use in 1984 reported lifetime use in 1988. In a follow-up study, Fendrich and Kim (2001) compared NLSY lifetime use reports provided in 1988, 1992, and 1994 with that reported in 1984. They found that 42% of the individuals who reported cocaine use and 29% of the individuals who reported marijuana use in 1984 did not report lifetime use at one or more of the follow-up surveys.
Stanton, Papandonatos, Lloyd-Richardson, and Niaura (2007) used data from the National Longitudinal Study of Adolescent Health to study the consistency of adolescents’ self-reports of lifetime cigarette use. Their participants constituted a representative sample of U.S. adolescents in grades 7 through 12. Stanton et al. found that 5.7% of the youths had reported some use at wave I and no use (“never smoked”) at wave III. Like Mensch and Kandel (1988), Stanton et al. found that the probability of providing a logically inconsistent self-report of lifetime use was greater among the respondents who had reported less frequent/intense smoking at wave I.
These survey-based studies demonstrate clearly that a percentage of youths’ self-reports of cigarette, marijuana, and cocaine use will exhibit logical inconsistencies when youths are assessed on multiple occasions. Longitudinal surveys are important because their study sizes are many times larger than those found in studies that assess convenience samples. More importantly, longitudinal surveys that collect data from nationally representative samples yield estimates that characterize the defined population. Unfortunately, these large surveys have had an important limitation. Because they are more expensive to field than the smaller, school-based studies, longitudinal surveys are less likely to be fielded many times within a 3- or 5-year period, less likely to assess individuals when they are experiencing a transition in use (e.g., the transition from having never used the substance to experimenting with the substance), and less likely to obtain data from a narrowly defined population (e.g., 10- to 11-year-old preadolescents).
Researchers who have sought to evaluate a substance use prevention intervention frequently analyze data reported by a convenience sample of youths participating in a school-based program. As one of the first school-based studies to report findings on the consistency of youths’ substance use reports, Collins, Graham, Hansen, and Johnson (1985) examined agreement between 415 high school students’ concurrent and retrospective self-reports of alcohol (specifically beer, wine, and liquor), tobacco, and marijuana use. The students were asked to complete questionnaires in January 1981 and May 1982 as part of a school-based, smoking prevention program. In Spring 1983, they were asked to recall the frequency of use that they had reported in 1981 and 1982. Agreement between the reports spaced 1 year and 2 years apart ranged from 74% (liquor) to 89% (marijuana). When Collins et al. examined the effect of the current level of use on the individual’s recall of earlier use, they found that the level of current use accounted for 50% to 71% of the explained variance among students whose reports of the frequency of use had differed. Moreover, Collins et al. found that the probability of providing a logically inconsistent self-report was greater among the respondents who had reported less frequent use. They concluded that students’ current substance use levels can bias their recall of past use and that this bias may be quite severe for students’ recall of past alcohol use.
For an evaluation of Project ALERT, Ellickson and Bell (1990) asked 7th- and 8th-grade students to complete four assessments administered over a 15-month period. On average, 5% of the middle school students reported no use after they had reported use earlier; less than 1% had reported less or no use after they had reported frequent use at an earlier wave. Like Collins et al. (1985), Ellickson and Bell found that the majority of the inconsistencies were reported by students who were experimenting with drugs or reported by students whose recall of when they had used the substance differed. Ellickson and Bell also found that participants in the treatment schools were no more likely to provide an inconsistent report than were participants in the control schools.
Finally, for their evaluation of Project DARE, Fendrich and Rosenbaum (2003) examined the consistency among students’ self-reports of lifetime alcohol, cigarette, marijuana, and cocaine use. Students completed a baseline assessment when they were in 5th/6th grade; they completed seven follow-up assessments as they advanced to the 11th/12th grade. When they analyzed data obtained from the students who participated in the baseline assessment and at least four follow-up assessments, Fendrich and Rosenbaum found that 45% of the students had provided inconsistent lifetime reports of alcohol use, 51% had provided inconsistent lifetime reports of cigarette use, and 81% had provided inconsistent lifetime reports of cocaine use.
These studies illustrate many of the interpretative problems reflected in the literature on the consistency of individuals’ self-reports of substance use. Specifically, researchers have not used the same cognitive task to assess the consistency. Some researchers ask students to report the frequency with which a substance was used; other researchers ask students to report lifetime use; and other researchers ask students to report particulars such as the age at which the substance was first used (see Table 1). Additionally, because evaluations of school-based, drug use prevention programs are based on data provided by students in middle, junior high, or high school and because researchers have asked these students to complete different cognitive tasks, it is difficult to organize the various findings on the consistency of students’ substance use self-reports. Moreover, although school-based studies are better situated to ask students more detailed questions about the cognitive strategies they may have used to respond to an item, few, if any, school-based studies have done so.
Stein et al. (2002) may have conducted the only study to ask adolescents why they might under- or over-report their substance use (specifically, smoking). The adolescents (n = 51) were recruited from a hospital outpatient clinic or emergency department and enrolled in a brief intervention designed to reduce their smoking. As participants, the adolescents completed a baseline assessment and three follow-up assessments that were conducted 1, 3, and 6 months after the baseline assessment. Stein et al. found that 3.9% of the youths had consistently under-reported their smoking, that 15.7% had sometimes under-reported their smoking, that 3.9% had sometimes over-reported their smoking, and that 3.9% had reported both less and more smoking than they had actually done. The two most prevalent reasons that youths gave for providing an inaccurate report were the youths’ perception that the interviewer wanted them to report less use (41.2%) and the youths’ perception that their parents wanted to know about the project (41.2%). Stein et al. concluded that researchers had to take steps to enhance participants’ beliefs that their responses would not be shared with others. Stein et al. also noted that researchers needed to establish rates of misreporting so that instruments could be evaluated properly.
Collectively, the studies summarized in Table 1 suggest the following four points. First, the researchers reported that the probability of observing an inconsistent lifetime substance use report increases with an increasing number of completed assessments. More assessments simply provide youths with more opportunities to give a subsequent report that is logically inconsistent with an earlier report. Moreover, if it is recognized that (a) some reporting errors are inevitable and (b) that a logically inconsistent lifetime use report has to be preceded by a report of lifetime use, and given the definition of prevalence (number of cases/number at risk), it follows that the probability of an inconsistent report would increase with an increasing number of reports. Second, the majority of the inconsistent reports are given by youths who are not frequent or regular users. Our understanding of cognition and memory tells us that the particular features that characterize a well-established behavior are easier to recall than are those that characterize a behavior we enact infrequently or enacted long ago (Johnson, Gerstein, & Rasinski, 1998). Third, the percentage of inconsistent lifetime substance use reports will fluctuate during the course of a multi-year study that has periodic assessments. Although the cumulative probability of an inconsistent report cannot decrease as the number of assessments increases, the marginal probability of an inconsistent report at the Tth assessment can be less than, equal to, or greater than the probability of an inconsistent report at an earlier assessment. A trend in the observed proportion of inconsistent reports across the assessments would suggest that a non-random process had been invoked and was in play (e.g., at the Tth assessment youths are capable of exercising the necessary care when they answer the substance use items). Finally, the cited studies suggest that the probability of an inconsistent lifetime substance use report is subject to demand characteristics and other social concerns (privacy, social stigma, legal sanction).
What this review of the literature does not convey is that inconsistent self-reports of lifetime substance use have been observed even when steps have been taken to catch these inconsistencies when the individual gave the inconsistent report. Indeed, in the National Survey on Drug Use and Health, inconsistencies were observed even when the data were collected using computer-assisted interviewing, when study retention criteria were used to drop respondents who did not provide data on lifetime substance use, and after data editing and imputation procedures were used to identify and resolve inconsistencies among variables related to key drug use measures (Substance Abuse and Mental Health Services Administration, 2007, pp. 122, 134).
The present study used data collected on six occasions that spanned a 40-month period to examine the consistency of Mexican and Mexican-American pre-adolescents’ self-reports of lifetime alcohol, cigarette, marijuana, and inhalant use. The youths were participating in a study that had been designed to evaluate two substance use prevention curricula for elementary school students. The first assessment was conducted when the youths were in 5th grade and most youths were 10 or 11 years old. The sixth assessment was conducted when the youths completed 8th grade and most were 14 or 15 years old. This 5-year period is a critical period when many youths initiate and experiment with substance use. As such, a study of youths’ perceptions and substance use behaviors during this period provides a strategic opportunity. In addition to taking advantage of its timing, the present study describes the pattern of reports provided by our participating preadolescents. Although previous studies have reported the aggregate rate of inconsistent reports observed when participants were assessed repeatedly, no other report to date has described the observed patterns or described how the rate of inconsistent reporting changes as respondents complete an additional assessment.
The present study focuses on Mexican and Mexican-American youths for three reasons. First, these youths constitute 71.5% of the youths who participated in the parent study. Second, and more importantly, our sample of Mexican and Mexican-American preadolescents were assessed during a critical period in their lives. Prior to 8th grade, the substance-specific prevalence rates observed for Hispanic youths are greater than those observed for African-American youths and roughly comparable to those observed for Caucasian youths. However, by 8th grade, the rates reported by Hispanic youths exceed those reported by African-American and by Caucasian youths (Johnston, O’Malley, Bachman, & Schlenberg, 2006). Third, unlike the survey-based studies that treat individuals with any Latino heritage as one homogeneous group (e.g., the Hispanic Health and Nutrition Examination Survey), the present study reports data provided by youths who self-identified as Mexican or Mexican American.
Twenty-nine public elementary schools in the Phoenix, Arizona metropolitan area agreed to participate in the evaluation of a NIDA-funded, school-based, substance use prevention intervention. The participating schools represented 7 of the 16 Phoenix school districts and were approximately half of the eligible elementary schools.
For their participation in the parent study, the 16 intervention schools received curriculum-specific teacher training and materials. Additionally, they received $500 to help defray the cost of implementing the intervention’s booster component. Students’ regular classroom teachers implemented the 12-lesson curriculum in the 5th and 7th grades; they supervised approximately five booster activities in the 6th and 8th grades. Curriculum implementation averaged 13 weeks for the 5th grade students and 16 weeks for the 7th grade students. Booster implementation averaged 18 weeks for the 6th grade students and 9 weeks for the 8th grade students. It should be noted that many of the intervention schools provided additional substance use programming to their students.
For their participation in the parent study, each of the 13 control schools received $1,000 for each year they participated in the parent study and permitted research staff to assess their students. Many control schools implemented substance use prevention activities or curricula.
Research assistants (RAs) initiated an active parental consent process and student recruitment by giving students consent forms to take home to their parents. Students received “kooky eggs” as an incentive to encourage them to return the consent forms to the RAs in a timely manner. The RAs obtained students’ assent before they administered each assessment. Although the teacher remained in the classroom as the questionnaire was administered, the RAs explained the importance of their maintaining confidentiality, encouraged the teachers to participate in other responsibilities (e.g., grading), and took precautions to ensure that the teachers did not view any student’s responses.
Because we anticipated that many of the 5th grade students would not be familiar with the concept of confidentiality or with certain phrasing and formats used with our questionnaires, we developed a detailed script for our RAs. Prior to obtaining a student’s assent, the RA stated:
Today you will be taking a survey for researchers at Arizona State University that asks questions about you, your family, your school, and about your experiences with drugs and alcohol. This is not a test. There are no right or wrong answers and no one will know how you respond to the questions. It will take about 45 minutes to complete all of the items.
After the RA had distributed blank assent forms and questionnaires to the students who had obtained a parent’s informed consent, the RA stated:
Please read the front page letter while I pass out the surveys. This survey is confidential. To make sure that it is not connected to any information about you please tear off the letter before we go on. The survey is voluntary (you can refuse to participate), and it is confidential. This means that no one will know how you answered the questions except you. You should be assured that it will be impossible to identify you.
The students then completed the assent forms and passed them to the RA, who put them into a separate envelope. As a final assurance of confidentiality, students were told that they should not put their names anywhere on the questionnaire and that:
Your answers will go to Arizona State University where the research staff will have the sheets scanned and then the surveys will be stored in locked file cabinets.
The RA then used a script and led the class in an exercise designed to take them through key questions. The latter included items used to collect data on students’ sociodemographic characteristics (sex, birth date) as well as items that the students might have difficulty answering. For example, to ensure a common understanding of the ethnicity items, the RA gave a verbal definition of ethnic group and then wrote the definition on the board. Finally, the RA also made the students aware of any “skip patterns,” particularly the pattern that the student was supposed to follow if the student had never used alcohol, cigarettes, marijuana, or inhalants. Collectively, these steps suggest that efforts were taken to ensure that the students were aware of our desire that they provide honest and complete responses.
The RAs administered six assessments. Wave 1 was administered from September 2004 to January 2005; Wave 2 was administered from February to June 2005; Wave 3 was administered from February to May 2006; Wave 4 was administered from September to December 2006; Wave 5 was administered from February to May 2007; and Wave 6 was administered from January to March 2008. At the beginning of the 2004 school year, 2,459 students were enrolled in the 5th grade and attending one of the 29 study schools that had agreed to participate in an evaluation of the parent study’s substance use prevention curriculum. Slightly less than 83% of these students returned signed parental consent forms. To date, 1,984 students have participated in one or more of the parent study’s assessments. The institutional review board of Arizona State University (ASU) approved all protocols and questionnaires. Additionally, ASU obtained a Certificate of Confidentiality from the National Institutes of Health.
At each wave, the data were collected with a questionnaire that was administered during a 45-minute classroom session and later scanned by project staff using Remark Office OMR software. Students could complete the questionnaires in Spanish or English. The percentage of students who identified as Mexican/Mexican American and completed the questionnaires in Spanish at each wave was 8.6%, 7.7%, 4.5%, 3.7%, 2.3%, and 1.8% respectively.
At the parent study’s baseline assessment, a dummy variable identified the students who were enrolled in a school that had been randomly assigned to an intervention condition (cond = 1) or the comparison condition (cond = 0).
We used two items at each wave to collect data on the student’s gender and race/ethnicity. Specifically, we asked students to fill in the circle next to the word to indicate whether they were a “boy” or a “girl.” We also asked students to mark all that applied to “What is your ethnicity?” The seven response choices were: “American Indian or Alaskan Native,” “African American or Black,” “Asian or Pacific Islander,” “White or Anglo,” “Mexican, Mexican American or Chicano,” “Some Other Latino or Hispanic Group,” and “Other ethnicity.” Students were included in the present study if they self-identified as “Mexican, Mexican-American, or Chicano” at any wave and had participated in at least two of the six assessments (n = 1,418). Students participating in the present study were 71.5% of the youths who participated in any assessment (n = 1,984).
We used two items at baseline and wave 4 to assess student’s propensity to take risks. In particular, students were asked to respond to the following two items: “Is it worth getting in trouble to have fun?” and “How often do you do things that adults tell you not to do?” Each item was scored on the following 5-point scale: 1 = “Never,” 2 = “Almost never,” 3 = “Sometimes,” 4 = “Often,” and 5 = “Always.” A scale score was calculated as the mean of the two item scores, and increasing scale scores indicated an increasing propensity to engage in risky behavior. Spearman’s rho for the present sample was 0.24 (n = 1088) and 0.47 (n = 677) at baseline and wave 4, respectively.
We used two items at each wave to assess a student’s perception of how many of her or his friends and schoolmates had ever used alcohol, cigarettes, or marijuana. The items were adapted from a study conducted by Hansen and Graham (1991). Specifically, students were asked “Now think about the friends you hang out with. How many do you think have used alcohol, cigarettes, or marijuana at least once?” and “About how many kids in your school would you guess have used alcohol, cigarettes or marijuana at least once?” Each item was scored on the following 4-point scale: 1 = “All or most,” 2 = “Half,” 3 = “Some,” and 4 = “Hardly any or none.” We reversed coded each item so that increasing item values would reflect an increasing prevalence among one’s friends and schoolmates. We then created four binary indicators (i.e., two indicators for each item). The indicator was set equal to 0 if students reported that they thought that “Hardly any or none” of their friends (kids in your school) “have used alcohol, cigarettes, or marijuana at least once.” The indicator was set equal to 1 if students reported that they thought that “Some,” “Half,” or “All or most” of their friends (kids in your school) “have used alcohol, cigarettes, or marijuana at least once.” This dummy coding scheme contrasted students who believed that few of their friends (kids in your school) had used alcohol, cigarettes, or marijuana at least once with students who believed that at least some of their friends had used alcohol, cigarettes, or marijuana at least once.
For the parent study’s baseline assessment, students were asked to use a 4-point Likert scale to indicate the extent to which they agreed with the statement “I was completely honest in filling out this survey.” The four response choices were 1 = “Strongly agree,” 2 = “Agree,” 3 = “Disagree,” and 4 = “Strongly disagree.” For the present study, item scores were reverse coded so that increasing values indicated stronger agreement and increasing confidence in the belief that one had provided completely honest responses.
We used four items at each wave to assess a student’s lifetime use of “Alcohol (beer, wine, and liquor),” “Cigarettes or tobacco,” “Marijuana (pot, weed),” and “Inhalants (sniff glue or paint).” The bubbles that students were to fill in for each substance they had tried appeared at the top of the page; they were aligned in one row and separated by a distance of 7/8 inch. The students were to select the “None of these” choice if it was the appropriate response and then follow the instructions for the indicated skip pattern. Students were classified as having used one or more of the four substances if they had filled in the bubble for any one of the four substances; students were classified as having never used one of the four substances if they selected the “none of these” choice or they did not fill in the bubble for any one of the four substances.
We used an additional four items at baseline and at wave 4 to assess the number of times a student had used one of the four substances in her or his lifetime. In particular, we asked “How many times have you drunk more than a sip of alcohol (beer, wine, or liquor) in your lifetime?” “How many times have you smoked cigarettes in your lifetime?” “How many times have you smoked marijuana (pot, weed) in your lifetime?” and “How many times have you sniffed glue, spray cans, paint, or other inhalants to get high in your lifetime?” Each item was initially scored on the following 7-point scale: 1 = “0 times,” 2 = “1–2 times,” 3 = “3–5 times,” 4 = “6–9 times,” 5 = “10–19 times,” 6 = “20–39 times,” and 7 = “40 or more times.” For the present study, we moved the origin to 0 (i.e., a value of 0 indicated 0 times) and created a 3-point scale where 1 = “1–9 times” and 2 = “10 or more times.” This coding allowed us to categorize students as non-users, light users, and heavier users; and assess the impact of the level of substance use on the probability of rescinding the corresponding lifetime substance use report.
Rodgers, Billy, and Udry (1982) used the term “rescission” to describe the inconsistent responses observed among adolescents’ self-reports of their sexual behavior. This term may be more appropriate than “deny” or “recant,” terms used by later researchers. Deny suggests that respondents were aware that their responses were logically inconsistent; the use of rescission conveys the understanding that the respondent’s inconsistent responses have called into question the researcher’s prevalence estimates and the conclusions that are based on those estimates. To assess participants’ rescission of their self-reports of lifetime substance use after baseline, we created five 0/1 binary variables. Each indicator was set equal to 1 if the student indicated that she or he had ever tried the substance at an earlier assessment and then did not do so for the current assessment; the indicator was set equal to 0 if the student did not indicate that she or he had ever tried the substance at an earlier assessment and did not do so for the current assessment. For the present study, we excluded 147 students who only participated in one wave and, therefore, could not provide an inconsistent substance use report.
When the research staff received a signed parental consent form, the staff assigned a unique identification number to the student and entered the student’s identifying information which included class, teacher, and school into a tracking database. This database could only be accessed by two research team members. On the six occasions when research liaisons went to the students’ classroom to conduct an assessment, they distributed questionnaires which had a student assent form attached to the front of each questionnaire. Before students completed a questionnaire, they completed an assent form. These forms (and the completed questionnaires) were collected separately and returned to the research office. Research staff used the information that students provided on the assent forms (first, middle, and last names; day and month of birth; updated contact information) to track the students during the 40-month study period. Additional tracking information was also obtained from the class rosters that the research liaisons obtained from school personnel. Discrepancies or questions about the information (e.g., slight variation in the spelling of a name) obtained from the signed parental consent form, the signed assent forms, and the class rosters were resolved by a research liaison who asked the student and/or teacher for clarification. In summary, the ASU research team members used repeated face-to-face contacts with school personnel and students to identify, verify, and track our study participants over the 40-month study period as they completed the six assessments, advanced from 5th grade to 8th grade, and changed classrooms and, in some cases, as they changed schools.
We used Stata’s programs for complex sample surveys (Stata, 2007) to obtain summary statistics (specifically means, proportions, and odds ratios) and their standard errors. These programs provided Student t-tests to assess mean differences and Chi Square tests to assess the homogeneity of proportions. (To account for the data dependencies associated with the students nested within schools, the test statistic reported for the test of homogeneity is an F statistic.) We used Stata’s complex sample survey logistic regression program (and a backward elimination procedure) to identify correlates of inconsistent reporting of lifetime alcohol, cigarette, marijuana, and inhalant use. Stata’s complex sample survey programs allowed us to account for the fact that we had obtained data from 29 intact groups of public, middle-school students. With intact social groups, we would not expect the observations reported by any two randomly selected students attending the same school to be statistically independent. Failure to account for this statistical dependence would result in standard errors that were underestimated, test statistics that were inflated, and p-values that were smaller than they would be if an appropriate statistical method had been used (Cornfield, 1978).
Roughly 40 months after administration of the parent study’s baseline assessment, the six waves of data reported by 1,418 urban, public, middle-school students who had self-identified as Mexican, Mexican American, or Chicano were consolidated. Slightly more than 32% of these students (n = 1,418) had participated in six assessments; 15.1%, 14.0%, 21.7%, and 17.1% had participated in five, four, three, and two assessments, respectively.
The number of students (and the percent female) at each wave were 1,390 (51%), 1,349 (50%), 1,120 (51%), 733 (53%), 688 (54%), and 722 (53%), respectively. A test of homogeneity of proportions indicated that female students, who were 50.5% of the 1,418 students, were as likely as male students to complete k = 2, …, 6 assessments (F(3.65, 102.13) = 2.11, p = .09). The decline in the number of students who completed an assessment during the course of the study reflects decisions made by the individual students and their parents, and by school administrators. Each year, between 9% and 23% of the students transferred from each school. Prior to wave 4, when the students were in 7th grade, three of the original 29 schools decided that they could no longer participate in the parent study and two additional schools were dropped from the parent study when the administrators declined to move to the scheduled “non-implementing” condition.
As Table 2 shows, the percentage of students reporting lifetime alcohol, cigarette, marijuana, and inhalant use increased with each wave. The rates of lifetime use were most similar for the first two waves, when students were in 5th grade, and then most similar for the next two waves, which were administered 14–18 months and 20–25 months after the baseline assessment, when students were in 6th and then 7th grade. Table 2 also shows that the rate of self-reported lifetime alcohol use greatly exceeded that of cigarette, marijuana, and inhalant use at each wave: By wave 6, slightly less than 60% of the youths who had completed two or more assessments reported that they had had more than a sip of alcohol (beer, wine, or liquor) in their lifetime.
Finally, Table 2 also displays the rates of inconsistent reporting associated with the students’ self-reports of lifetime alcohol, cigarette, marijuana, and inhalant use. Again, each value is based on the number of students who participated in two or more assessments and who participated in that wave. All students were scored as “consistent” at wave 1 and “inconsistent” if any later report was logically inconsistent with any earlier report. By wave 6, almost one-fourth of the students had rescinded an earlier report of lifetime alcohol use.
Table 3 summarizes the patterns of lifetime reports for the patterns with one or more inconsistent reports. (There were 35 = 243 possible sequences as a student could report lifetime use (Yes), no lifetime use (No), or not provide data (missing) at any assessment.) With six waves, the most complex of the observed patterns was “YNYNYN,” which indicates that the student initially reported lifetime use and then gave a report that was inconsistent with the preceding assessment. The simplest sequence was an isolated “YN” (i.e., a two character sequence preceded by one or more Ys and/or ms for missing, and succeeded by Ns or ms): this sequence indicates that the student reported lifetime use and then did not report lifetime use at the next assessment. The “YNYNYN” sequence was only observed for students’ self-reports of lifetime alcohol use: although 24% of the students (n = 1,418) gave an inconsistent alcohol use report, only one student alternated between use and no use.
Two patterns accounted for the majority of the inconsistent lifetime substance use reports. For the most prevalent pattern, the student reported lifetime use and then did not report lifetime use at the next assessment (i.e., one “YN” sequence among four remaining characters). For the second most prevalent pattern, the student provided two inconsistent reports and then missed the next assessment (i.e., “YNm”). These two patterns accounted for 79% of the inconsistent alcohol use reports, 82% of the inconsistent cigarette use reports, 84% of the inconsistent marijuana use reports, and 73% of the inconsistent inhalant use reports. Collectively, patterns with at most one “YN” (i.e., one inconsistent) sequence accounted for 91% of the inconsistent alcohol use reports, 92% of the inconsistent cigarette use reports, 93% of the inconsistent marijuana use reports, and 89% of the inconsistent inhalant use reports.
Having assessed lifetime substance use with (a) four yes/no items at each wave and (b) four frequency of lifetime use items that were assessed at baseline and at wave 4, we used the two sets of items to assess the rate of inconsistent reporting of lifetime alcohol, cigarette, marijuana, and inhalant use within the baseline and wave 4 assessments. At baseline, the rates of inconsistent reporting based on the four yes/no items and the four lifetime frequency items were 2.5% (alcohol), 2.2%, 1.3%, and 4.6% (inhalant), respectively; at wave 4, the rates were 4.8%, 5.4%, 3.0%, and 8.3%. Clearly, our students were far less likely to provide inconsistent lifetime substance use reports when they responded to different lifetime substance use measures that were on the same questionnaire; they were far more likely to provide inconsistent reports when they responded to the same lifetime substance use items on successive questionnaires.
Finally, we fit four logistic regression models to determine if we could identify variables associated with the odds that a student had provided an inconsistent report of lifetime alcohol, cigarette, marijuana, and inhalant use. Each model included the student’s: gender; age at baseline; parent study condition (intervention, control); number of completed assessments; the student’s assessment at baseline of how completely honest he or she had been in answering questionnaire items; the student’s perception at baseline of her or his propensity for engaging in risky behavior; a binary indicator for the student’s perception at baseline of how many friends “used alcohol, cigarettes, or marijuana at least once”; a binary indicator for the student’s perception at baseline of how many “… kids in [the] school … used alcohol, cigarettes, or marijuana at least once”; and a trichotomous indicator for the student’s baseline report of the number of times the student had used the substance. After fitting a saturated model, we used backward elimination to obtain the reduced models presented in Table 4.
As Table 4 indicates, the log odds of an inconsistent substance use report were positively related to the number of completed assessments: the more assessments a student had completed, the greater the odds that the student had given an inconsistent report of lifetime use. Moreover, with the exception of alcohol use, we found that there was a statistically significant relation between the number of times the student had ever used the substance at baseline (“no use,” “1–9 times,” and “10 or more times”) and the log odds that the student would give an inconsistent report of lifetime use for the substance. Linear contrasts on the mean predicted probability of an inconsistent substance use report indicated that the predicted probability of an inconsistent lifetime report increased with the number of times the students reported that they had ever used the substance at baseline.
It should be noted that the log odds of an inconsistent lifetime substance use report were not related to the students’ perception of their willingness to engage in risky behavior or their perception of how many of their friends or schoolmates “have used alcohol, cigarettes, or marijuana at least once.” Moreover, the log odds of inconsistent lifetime substance use report were not related to how honest students believed that they had been. At baseline, 70% of the male students and 76% of the female students agreed or strongly agreed that they had been completely honest. (Twenty-one of the students participating in the baseline assessment did not provide data for the honesty item.) The students who had provided inconsistent lifetime alcohol, marijuana, and inhalant use reports believed that they had been as honest as the students who had provided consistent reports. However, the students who provided inconsistent lifetime cigarette use reports perceived that they had been less candid (m = 3.3) than the students (m = 3.6) who provided consistent lifetime use reports (Δ = .28, SE = .095, t(28) = 2.92, p = 0.007).
We examined urban, public, middle-school students’ self-reports of lifetime alcohol, cigarette, marijuana, and inhalant use that had been assessed repeatedly over a 40-month period. The students joined the parent study when they were in 5th grade. They were included in the present study because they had self-identified as Mexican or Mexican American and had participated in two or more assessments. The latter were conducted during the Fall of 2004 (baseline) and 4–7 months, 14–18 months, 21–25 months, 27–30 months, and 38–40 months after the baseline assessment. All four self-reported lifetime use rates reached their maximum value at wave 6, when the students were in 8th grade. The increasing prevalence rates observed among our participants as they negotiated the transition from elementary school to middle school is consistent with recent research on the patterns of use among African-American, Caucasian, and Hispanic youths (Johnston et al., 2006).
At each assessment and when averaged across the six assessments, the rate of self-reported lifetime alcohol use greatly exceeded that reported for cigarettes, marijuana, and inhalants. This finding suggests that alcohol was much more available and/or that the costs (economic, social, lost opportunities) associated with attaining and consuming alcohol use were much less than the costs associated with the other substances. The rate of lifetime marijuana use started to differ from the rates of lifetime inhalant and cigarette use by wave 3. After wave 3, the self-reported lifetime marijuana and cigarette use rates increased steadily while the inhalant use rate remained relatively flat.
With respect to the rescission of a lifetime substance use report, our Mexican/Mexican-American preadolescent males were as likely as their female counterparts to rescind an earlier lifetime alcohol, cigarette, or marijuana use report. This finding is consistent with the finding reported by Johnston and O’Malley (1997) who used data reported by repeated cross-sectional surveys of nationally representative samples.
Our finding that the risk of an inconsistent lifetime substance use report increased with the lifetime frequency of use that the preadolescent students reported at baseline may be different from the findings reported by Collins et al. (1985), Mensch and Kandel (1988), and Stanton et al. (2007). The latter found that the probability of an inconsistent lifetime substance use report was greatest among the participants who initially reported the least use and who were thought to be experimenting. However, we found that the probability of an inconsistent lifetime substance use report was greatest among the participating Mexican and Mexican-American preadolescents who reported the greatest number of substance use occasions at baseline, not the least number of occasions. The different findings may be explained in part by the relatively young age of our study participants, by the fact that our heaviest lifetime use category was “more than 10 times,” and by the fact that researchers do not have widely-accepted definitions regarding the number of drug use occasions that define experimentation, occasional use, and regular use.
Our finding that the risk of an inconsistent lifetime substance use report increased linearly with the number of completed assessments is consistent with the finding reported by Fendrich and Rosenbaum (2003), and undoubtedly reflects the fact that some students simply had more opportunity to rescind an earlier report. For these students, we do not know whether they simply did not recall the occasion that resulted in their reporting that they had used the substance on the earlier assessment. For some students, their inconsistent reporting may reflect their coming to a different definition, for example, of their having ever had a sip of alcohol or having ever tried a cigarette.
Finally, our finding that the majority of the inconsistent reports of lifetime substance use could be described by an isolated “YN” pattern that was not preceded or followed by another “YN” combination suggests that the majority of the inconsistencies were not due to a student’s intentionally trying to misreport his or her lifetime use of a substance. Moreover, our findings that (a) the majority of our participants believed that they had provided honest reports and (b) that students’ rating of their honesty in answering the questionnaire items was not consistently associated with the probability that they had rescinded an earlier report of lifetime substance use were consistent with findings reported by Siegel, Aten, & Roghmann (1998). The latter used two items to assess the honesty of students’ self-reports of their sexual behavior (specifically, whether they had ever had sexual intercourse and the frequency of intercourse). The students were participating in an intervention designed to prevent pregnancy and sexually transmitted infection. Although 78% of their middle school males (n = 793) and 98% of their middle school females (n = 825) reported that they had been very or completely honest, 14% of the males acknowledged that they had overstated their actual behavior, and 8% of the females acknowledged that they had understated their behavior. When Siegel et al. analyzed the associations between students’ ratings of their honesty and four reported sexual behaviors, they found that the associations were most consistent among the students who had understated their behavior.
Although random assignment of schools to study condition provides a basis for drawing inferences in our parent study, the youths participating in the present study constitute a convenience sample of Mexican and Mexican-American youths attending a public middle school in Phoenix, Arizona. As such, we do not claim that any of the reported sample statistics are unbiased estimates of their nominal population parameters. As with any observational study, the reported sample statistics may reflect an unobserved bias of unknown magnitude.
Henry (2007) analyzed self-report data collected on 8th- and 10th-grade students who were participating in the 2003 wave of Monitoring the Future. She found that 10.5% of 8th-grade students and 16.4% of 10th-grade students reported that they had skipped school within the past month. More importantly, she found that students who were disengaged from school and who reported that they had used drugs within the past month were the most likely to report a recent truancy. If Henry’s findings were to hold for our middle school students, we would expect that a proportion of the youths who could not be assessed because they had skipped school on the day a questionnaire was administered were using drugs. Finally, we may not have detected correlates of Mexican/Mexican-American preadolescents’ inconsistent reporting of their lifetime substance use because 5 of the 29 schools participating in the parent study had withdrawn from the latter by the 6th and final wave. Although the loss of their students undoubtedly made it more difficult to detect a variable that predicted a missing response, it also made it more difficult to identify an individual-level characteristic that may have been correlated with inconsistent reporting.
Rescission rates for lifetime alcohol, cigarette, marijuana, and inhalant use among our sample of Mexican/Mexican-American, urban, public, middle school students differed across the four substances. The rescission rate was 9.2% for lifetime inhalant use, 9.6% for lifetime cigarette use, and 5.8% for lifetime marijuana use. Offsetting these lesser rates, the rescission rate for lifetime alcohol use during the 40-month period was 24%. For each substance, the rescission rate kept pace with the corresponding lifetime substance use rate reported for its assessment period. As noted earlier, the fact that the rescission rate observed for each follow-up assessment increased as the reported lifetime substance use rate increased is not surprising as the youths had to report use before they could rescind an earlier report. More importantly, if the number of inconsistent reports is proportional to the number of individuals reporting use of the substance at time T, then the number of inconsistent reports at time T would be a function of the prevalence at time T and the number of individuals providing a report at time T.
On balance, the present study found that the majority of preadolescent Mexican and Mexican-American youths who are participating in an evaluation of a substance use prevention intervention will report lifetime substance used in a manner that is logically consistent over time when they are assured that their data will be treated in a confidential manner. Compared to past studies of inconsistent substance use reporting, the present study is unique in the following three respects. First, it analyzed self-report data collected from public middle school students who self-identified as Mexican or Mexican American. Students who identify as Mexican American are projected to become the majority in many public schools in the United States. Indeed, Mexican-American students were the majority in many of the parent study’s participating schools. Second, the present study analyzed data collected through six assessments that were conducted over a 40-month period. Past studies have been largely limited to two or three assessments which necessarily limit the potential for observing inconsistent substance use reports. Third, and most importantly, the present study appears to be the only study to describe the various patterns of inconsistent reporting that were observed. This observation is surprising as response patterns are used to define editing rules and address logical inconsistencies in respondents’ data.
The data used in the present study would not have been available had it not been for the dedication of the Drug Resistance Strategies Project team members in Phoenix, Arizona. These researchers were led by Drs. Flavio Marsiglia, Stephen Kulis, and Patricia Dustman. Finally, we would like to thank Dr. Heather Cecil and Dr. Tanya Nieri for their helpful comments and suggestions.
This publication was supported by Grant Number (RO1 DA005629) from the National Institute on Drug Abuse to The Pennsylvania State University (Michael Hecht, Principal Investigator). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.
DAVID A. WAGSTAFF, The Pennsylvania State University.
STEPHEN KULIS, Arizona State University.
ELVIRA ELEK, Research Triangle International.