PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Res Adolesc. Author manuscript; available in PMC 2010 June 8.
Published in final edited form as:
J Res Adolesc. 2010 June 1; 20(2): 274–286.
doi:  10.1111/j.1532-7795.2010.00648.x
PMCID: PMC2882312
NIHMSID: NIHMS174174

A Tobit Regression Analysis of the Covariation between Middle School Students’ Perceived School Climate and Behavioral Problems

Abstract

This study uses an ecological framework to examine how adolescents’ perceptions of school climate in 6th grade covary with the probability and frequency of their engagement in problem behaviors in 7th and 8th grades. Tobit analysis was used to address the issue of having a highly skewed outcome variable with many zeros and yet account for censoring. The 677 participating students from 8 schools were followed from 6th through 8th grade. The proportions of students reporting a positive school climate perception decreased over the middle school years for both genders, while the level of problem behavior engagement increased. The findings suggested that students who perceived higher levels of school discipline and order, or more positive student–teacher relationships were associated with lower probability and frequency of subsequent behavioral problems.

The incidence and frequency of adolescent problem behaviors is a major concern because adolescents who manifest problem behaviors are also more likely to exhibit negative educational outcomes (i.e., low grades, and peer rejection) which often lead to school failure (Gutman, Sameroff, & Cole, 2003). From an ecological perspective, human behavior always occurs within multiple contexts, making reciprocal interactions between individuals and their environments a crucial area of inquiry (Bronfenbrenner, 1979, 1986). If we wish to support healthy adolescent development, we must foster the school environment within which the adolescent interacts instead of only addressing the selected characteristics of the individual adolescent. During the transition to middle school, many adolescents confront a series of new social and educational demands that place some of them at greater risk for the development of problem behaviors (Eccles & Midgley, 1989). To appropriately identify and prevent the development of risk factors associated with problem behaviors, researchers and practitioners need an overarching understanding of what school characteristics either exacerbate or ameliorate that risk. To examine these characteristics, this study measures four dimensions of the school climate as perceived by adolescents: academic focus, discipline and order, peer relationships, and student–teacher relationships, and relates these perceptions to their self reported problem behavior.

There is considerable evidence that the academic focus, discipline and order, and social climate of middle schools significantly influence students’behavioral and emotional outcomes (e.g., Kuperminc et al, 2001; Roeser, Eccles, & Sameroff, 2000). Academic focus refers to a serious and orderly climate in which teachers believe students can achieve, and students work hard to succeed. Students in schools with a strong academic focus are more likely to be academically motivated and have fewer student behavioral problems (Weishew & Peng, 1993). Discipline and order involves the degree to which students behave well in school and adults treat students in a fair and equitable manner. Students who report that their schools establish, communicate, and enforce a fair discipline system with clear rules and consequences have fewer problem behaviors and avoid victimization (Gottfredson, 2001; Sugai & Horner, 1999). Additionally, the social climate of the school, including peer relationships and student-teacher relationships, plays an essential role in the development of student problem behaviors. Students who perceive that teachers are supportive, responsive, and care about them have better achievement and fewer behavior problems (e.g., Loukas & Robinson, 2004; Reinke & Herman, 2002). Similarly, peer acceptance and positive peer interactions have been positively associated with adolescents’ behavioral and emotional problems (Stewart, 2001).

Most existing research in school climate and problem behaviors have used ordinary least squares regression to analyze the data. However, in these studies, a majority of adolescents reported a lack of problem behaviors during the time period under study, so that the highly skewed distribution of the outcome variable includes many students with a zero value. This distributional property makes it difficult to justify the use of ordinary least squares regression because of the violation of the normal theory assumptions and can produce biased and inconsistent parameter estimates (e.g., Grogan-Kaylor & Otis, 2003). Researchers usually appear to have dealt with this problem by log-transforming the dependent variable (e.g., Kuperminc et al., 2001; Loukas & Robinson, 2004). However, transforming a highly skewed variable with many zeros results in a loss of information and is problematic because no transformation can separate the zeros from each other, normalizing the distribution. We used Tobit regression in this study to address the issue of having a highly skewed outcome variable with many zeros. Moreover, the study’s measure of problem behavior records the number of times adolescents had engaged in problem behaviors in the past 3 months. Adolescents who had experienced problem behaviors earlier than 3 months ago, but not in the past 3 months, are treated as adolescents without problem behaviors. The measure of problem behavior may result in a variable that is left censored at zero by the data collection process. Tobit analysis accounts for the censoring and allows for the estimation of the impact of independent variables on the uncensored variable (Greene, 1997).

Previous research examining predictors of adolescent problem behavior has focused primarily on how perceived school climate relates to differences in the frequency with which adolescents engage in problem behaviors (Broidy et al., 2003). Little attention, however, has been given to ways that school climate may influence the very existence of these problem behaviors. With early intervention, there is greater chance of preventing or postponing the first occurrence of the problem behaviors (Kellam & Langevin, 2003). Thus, prevention programs for adolescents have a strong interest in reducing both the incidence and frequency of problem behaviors by adolescents simultaneously. In this study, Tobit regression analysis was used for addressing the methodological issues previously described and for obtaining estimates of the impact of key predictors on both the occurrence and frequency of an adolescent’s engagement in problem behaviors (McDonald & Moffitt, 1980; Tobin, 1958).

The study applied an ecological framework to examine the link between school climate and development of problem behaviors in the microsystem level of schools (Bronfenbrenner, 1986). Using a three-wave longitudinal data set, our research focuses on how students’ perceptions of school climate while they are in sixth grade, both overall and in its four particular dimensions (academic focus, school discipline, peer relationships, and student–teacher relationships), predict their engagement in problem behaviors in the seventh and eighth grades. In particular, Tobit regression analysis allowed us to explore how perceptions of school climate relate to the probability and frequency of problem behaviors. Furthermore, given prior research showing the lack of conclusive findings about gender differences in terms of responding to the school climate, it examines how the relations between students’perceptions of school climate and problem behaviors may differ as a function of gender (e.g., Kuperminc et al., 1997).

Method

Research Questions

  1. Do students with more positive perceptions of school climate while they are in sixth grade have lower probability of engagement in problem behaviors in seventh and eighth grades?
  2. For those who engage in problem behaviors, do students with more positive perceptions of school climate while they are in sixth grade engage in fewer problem behaviors in seventh and eighth grades?
  3. Does the students’gender moderate the relations between students’current perceptions of school climate and subsequent problem behaviors?

Sample

Our data were collected as part of the Next Generation Project, which was designed to understand how middle school parents and schools can work together to promote success, health, and well-being in the next generation of youth (DA 13773; Stormshak & Dishion, 2002). The Next Generation Project began by recruiting students from eight middle schools in a school district system in the Pacific Northwest. Participating students were followed from sixth through eighth grades. For the purposes of this study, we only included those students who participated in three consecutive grades of the study in the sample, for a total of 677 participants. The drop-out rate across the three years was 9%, 11%, and 13% respectively. Approximately 76% of participating students were European American, 5% Asian or Pacific Islander, 4% Hispanic, 1% African American, and 14% others. Approximately 54% of the students were female. The percentage of students on free or reduced lunches is 25% across the 8 middle schools. To ascertain whether the students who dropped out of the study in any waves differed from the students who participated in all three waves, a series of t-tests were conducted with all study variables at each wave. Results revealed that those who participated in the study for all three waves were more likely to be girls and to have fewer problem behaviors than were those who dropped out of the study in any waves.

Measures

Outcome Variable

Student self-reports of problem behavior in seventh and eighth grades

Student responses to nine self-report items from the Social Skills Rating System (Gresham & Elliott, 1990) were averaged to assess the extent of externalizing problem behaviors on each occasion, with higher scores reflecting more problem behavior. Each question is answered using a 6-point scale, ranging from 1 (never) to 6 (more than 20 times). Example items included “In the last three months, how many times have you carried or handled a weapon?”, “How many times have you skipped school without excuses?”, and “How many times have you stolen or tried to steal things?” The measure had high estimated internal consistency reliability (see table 1) and has been used in previous research with at-risk youth to measure problem behavior (Biglan, Metzler, & Ary, 1994).

Table 1
Descriptive Statistics and Multivariate Analyses of School Climate Perceptions and Problem Behaviors from Grade 6 through Grade 8 (N = 677)

Additionally, we selected five similar items assessing students’ problem behaviors from student report and teacher report data to examine whether the teachers would be consistent with what the students were reporting on themselves. We found there was no statistically significant difference in the problem behaviors between student report and teacher report. Thus, this indicates that the self-reported problem behaviors from students were reasonably reliable.

Question Predictor

School climate perceptions in sixth grade

Seventeen items were chosen from Dishion and Stormshak’s (2002) Social Nomination Measure to assess adolescents’ perceptions of school climate, including four subscales of the quality of school climate: (a) academic focus, (b) discipline and order, (c) peer relationships relationships, and (d) student-teacher relationships. Each subscale is a composite of two to six questions. All the questions are written in common format and ask students questions such as “How many of the students in your class work to complete homework?”, “How many of the students in your school understand the consequences of violating the rules?”, “How many of the students in your school are friendly to you?”, and “How many of the teachers in your school treat students with respect?” Each question is answered using a 6-point scale, ranging from 1 (none) to 6 (almost all). High scores for each subscale indicate more positive student-perceived school climate. Student responses to items in each subscale were averaged to create a subscale score, and then an overall school climate variable was constructed by combining student responses on all 17 of the items. The overall school climate construct and the four subscales of school climate demonstrated high internal consistency reliability (see table 1).

Covariates

Students’ prior problem behavioral status in sixth grade and demographics was controlled in the statistical models as covariates. Demographic characteristics of the target sample included student ethnicity, gender, and socioeconomic status (SES). The SES indicator was a school level variable which indicated the percentage of students on free or reduced lunches across the 8 middle schools.

Data Analyses

In the Tobit regression analyses, the impact of school climate on student problem behavior was investigated, controlling for selected covariates. To examine the moderating effects of gender in these processes, a two-way interaction of gender was included in the model. Strategies proposed by McDonald and Moffitt (1980) were used to decompose the parameter estimates into two parts. The first part summarizes whether or not participants possess a nonzero value on the dependent variable, and describes the incidence of adolescents’engagement in problem behaviors. The second part of the decomposition summarizes the effect of the independent variable on the outcome for participants with a nonzero value on the dependent variable. This summarizes the frequency of adolescents’ problem behaviors for those who do experience them. Additionally, students in the same school could have common unobserved experiences as a result of their membership in the original school structure and thus possess a nonzero intraclass correlation. Thus, we fitted our multilevel modeling procedure in STATA using random-effects models that account for the nested nature of our data (students within schools). In the models, the random effect of school was included to account for the potential dependence of residuals among students within the same school.

Results

Descriptive Statistics

To assess change over time, a series of multivariate analyses was conducted on the measures for school climate and problem behaviors. The model consisted of one within-subject factor (time: sixth, seventh, and eighth grade) and one between-subject factor (gender). The analyses in table 1 revealed significant effects of gender and time on both school climate perceptions and problem behaviors. Consistent with past research (Esposito, 1999; Simons-Morton et al., 1999), on average, middle school students’ perceptions of school climate became negative over time, but these negative changes did not differ by gender. Although boys consistently reported higher levels of problem behavior than did girls, the level of engagement in problem behaviors increased from sixth through eighth grade for adolescents of both genders. All the correlations between perceptions of school climate and problem behaviors during sixth grade were significant, and all were in expected directions. That is, lower levels of problem behavior were associated with more positive school climate perceptions. The same pattern of relations, though slightly stronger in magnitude, was found during both seventh grade and eighth grade.

Effects of School Climate Perceptions on Adolescents’Subsequent Reports of Whether or Not They Engage in Problem Behaviors (First Decomposition of Tobit Coefficients)

The first decomposition of the fitted Tobit coefficients summarizes the extent to which each independent variable predicted the probability that an adolescent would or would not engage in problem behaviors (see column 1 & 3 in table 2 & 3). Students who perceived a more positive school climate in sixth grade had a lower probability of experiencing problem behavior in seventh grade and in eighth grade. For example, when students’ perceptions of school climate are one standard deviation higher in sixth grade, it was associated with a 7% decrease in the probability of engagement in problem behavior in seventh grade. Furthermore, the four dimensions of school climate perceptions simultaneously were entered as predictors, replacing overall school climate to determine what aspects of school climate support the reduction of adolescents’ problem behaviors (see column 3 in table 2 & 3). The results indicated that positive student–teacher relationships and discipline and order were more substantially associated with lower probability that an adolescent would engage in problem behavior. For instance, for students in seventh grade, one standard deviation higher in students’ perceived student–teacher relationships was associated with a 4.0% decrease in the probability of engagement in problem behavior, and one standard deviation higher in students’ perceived school discipline and order was associated with 6.0% decrease in the probability of engagement in problem behavior. The interaction of perceptions of school climate with gender was not statistically significant.

Table 2
Decomposition of Tobit Coefficients: Probabilities that Adolescents will Experience Problem Behaviors and the Frequency of Problem Behaviors in Grade 7 (N = 677)
Table 3
Decomposition of Tobit Coefficients: Probabilities that Adolescents will Experience Problem Behaviors and the Frequency of Problem Behaviors in Grade 8 (N = 677)

Effects of School Climate Perceptions on Frequency of Engagement in Problem Behavior (Second Decomposition of Tobit Coefficients)

The second decomposition of the fitted Tobit coefficients summarizes the relationship of the school climate perceptions with the frequency of problem behavior experienced by adolescents in the past 3 months, conditional on those adolescents having experienced one or more incidents of problem behavior (see column 2 & 4 in table 2 & 3). As with the previous analyses, for those who had experienced these problem behaviors, students who perceived a more positive school climate while in sixth grade had a lower frequency of engagement in problem behaviors in seventh and eighth grade. For instance, when students’ perceptions of school climate are one standard deviation higher in sixth grade, it was associated with a 0.05 decrease in the frequency of engagement in problem behavior in seventh grade. In comparison with other indicators of school climate perceptions, students who perceived higher levels of school discipline and order, or more positive student–teacher relationships, were more strongly associated with lower levels of engagement in problem behavior. For example, one standard deviation difference in students’ perceived discipline and order was associated with a −0.04 difference in the frequency of engagement in problem behavior in seventh grade. There was no moderating effect of gender in these processes.

In sum, school discipline and order and student–teacher relationship are associated with the probability and frequency of adolescents’subsequent problem behaviors more than are other two dimensions of school climate (peer and academic factors). The associations were greater in magnitude in eighth grade than in seventh grade.

An additional analysis was conducted to examine the reciprocal effects that students’ high levels of problem behaviors may lead to negative school climate perceptions. To address this possibility, we used students’ problem behaviors in grade 6 to predict their school climate perceptions in grade 7 and 8. The Tobit regression equation was identical to that previously described except for the reverse of predictors and outcomes. We found there were no statistically significant effects of problem behaviors in grade 6 predicting school climate perceptions in grade 7 or in grade 8, after controlling for school climate perceptions in grade 6. Thus, changes in school climate perceptions from one grade to the next grade cannot be attributed to prior levels of problem behaviors.

Discussion

The results of this study are consistent with past research that students’perceptions of school climate are associated with the frequency of adolescent engagement in problem behavior for those who had already experienced such behaviors (e.g., Esposito, 1999; Loukas & Robinson, 2004). This study adds to the research base by illuminating the extent to which students who have more positive perceptions of school climate have lower probability that they would experience problem behaviors in the future. Thus, the findings provide a better basis for schools to develop school-based interventions that focus not just on reducing the frequency of problem behaviors by adolescents, but also on preventing adolescents from engaging in problem behaviors in the first place. Because adolescents’school climate perceptions are strongly predictive of problem behavior, the findings highlight the importance of creating positive and supportive school climates in order to prevent and reduce engagement in problem behaviors when adolescents enter middle school. With limited school resources, a focus on two specific dimensions (school discipline and order and student–teacher relationships) may be more beneficial for students than would a focus on the other two dimensions to improve students’ behavioral adjustment and reduce problem behavior.

Furthermore, the decomposition of the Tobit coefficients suggested that it is easier to predict the likelihood of engagement in problem behavior than the frequency with which adolescents experience problem behavior. The effect sizes of the parameters associated with likelihood were larger than those associated with frequency. For example, for students in the eighth grade, an increase of one standard deviation in adolescents’ perceptions of school climate was associated with an approximately 8% decrease in the likelihood of adolescents engaging in problem behavior. However, for adolescents who already engage in problem behavior, a one standard deviation increase in adolescents’ perceptions of school climate was associated with only a 0.06 decrease in the frequency of problem behavior. Thus, interventions targeted at improving adolescents’perceptions of school climate may have a greater effect on preventing adolescents from engaging in problem behaviors than it does on reducing the frequency with which adolescents repeatedly engage in the problem behavior. These findings suggest that school interventions that improve school climate might reduce the subsequent incidence of problem behaviors more effectively by preventing these behaviors before they occur, than by focusing on reducing the number of repeated occurrences.

Limitations

This study has some limitations that merit consideration. First, it limits itself to understanding student perceptions alone, and relies upon self-report information from students to assess perceptions of school climate which might raise an important validity concern (McEvoy & Welker, 2000; Roeser & Eccles, 1998). Social desirability problems may be operating in that the preferred behavior influences the students’perceptions of school climate. The future use of multiple sources of information about the school environment (e.g., teachers, and parents) and multiple methodologies (e.g., interview and surveys) can provide a more robust, valid method for identifying school effects (Gottfredson, & Gottfredson, 2001). Second, this study’s sample comprises primarily European American middle school students and indicates very little racial diversity. Therefore, these findings may be limited in their generalizability to more diverse populations. Finally, the non-experimental nature of the study limits our ability to make causal inferences. The effect of students’perceptions of school climate was not exogenous and therefore the effect might be affected by other observed and unobserved variables.

This study has provided a stronger methodological basis that will lead to improvements in related research by others, and we hope this will in turn provide a solid foundation for future theoretical advances. The findings also confirm the importance of creating school settings that advance school climates that are perceived by students as orderly, structured places where all students behave appropriately and have positive student–teacher relationships. Creating such an environment improves both the quality of students’ educational experiences, and also reduces and prevents later involvement in problem behaviors. By understanding how a positive school climate can serve as a protective factor against subsequent problem behaviors, schools can work more effectively in developing new initiatives that will improve student perceptions of school climate.

Acknowledgments

We thank Dr. John Willett for his contributions to this manuscript regarding statistical analysis issues. We also thank Dr. Terry Tivnan, Cher Mikkola, Amy Fowler, Rebecca Holcombe, and Thomas Nikundiwe for their feedback on this manuscript. This project was supported by grants DA 018760, DA 07031, and DA 13773 from the National Institute on Drug Abuse at the National Institutes of Health to Thomas J. Dishion, PhD.

Contributor Information

Ming Te Wang, Harvard University.

Robert L. Selman, Harvard University.

Thomas J. Dishion, Child and Family Center, University of Oregon.

Elizabeth A. Stormshak, Child and Family Center, University of Oregon.

References

  • Biglan A, Metzler CW, Ary DV. Increasing the prevalence of successful children: The case for community intervention research. The Behavior Analyst. 1994;17:335–351. [PubMed]
  • Broidy LM, Tremblay RE, Brame B, Fergusson D, Horwood JL, Laird R, et al. Developmental trajectories of childhood disruptive behaviors and adolescent delinquency: A six-site. Cross-National Study. Developmental Psychology. 2003;39(2):222–224. [PMC free article] [PubMed]
  • Bronfenbrenner U. The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press; 1979.
  • Bronfenbrenner U. Ecology of the family as a context for human development. Developmental Psychology. 1986;22:723–742.
  • Centers for Disease Control and Prevention. Chronic disease and health promotion reprints from the MMWR: 2000 Youth Risk Behavior Surveillance System. USDHHS PHS, NCCDOHHP; 2001.
  • Dishion TJ, Stormshak EA. Social Nominations. Child and Family Center; 195 W.12th Ave., Eugene, OR 97401: 2002. Unpublished measure.
  • Eccles JS, Midgley C. Stage/environment fit: Developmentally appropriate classrooms for early adolescents. In: Ames RE, Ames C, editors. Research on motivation in education. Vol. 3. New York: Academic Press; 1989. pp. 139–186.
  • Esposito C. Learning in urban blight: School climate and its effect on the school performance of urban, minority low-income children. School Psychology Review. 1999;28(3):365–377.
  • Gottfredson DC. Schools and delinquency. New York: Cambridge University Press; 2001.
  • Gottfredson GD, Gottfredson DC. What schools do to prevent problem behavior and promote safe environments. Journal of Educational and Psychological Consultation. 2001;12(14):313–344.
  • Greene WH. Econometric analysis. 3. Upper Saddle River, NJ: Prentice Hall; 1997.
  • Gresham FM, Elliot SN. Social Skills Rating System manual. Circle Pines, MN: American Guidance Service; 1990.
  • Grogan-Kaylor A, Otis MD. The effect of childhood maltreatment on adult criminality: A Tobit regression analysis. Child Maltreatment. 2003;8(2):129–137. [PubMed]
  • Gutman LM, Sameroff AJ, Cole R. Academic growth curve trajectories from 1st grade to 12th grade: Effects of multiple social risk factors and preschool child factors. Development and Psychopathology. 2003;39:777–790. [PubMed]
  • Kellam SG, Langevin DJ. A framework for understanding “evidence” in prevention research and programs. Prevention Science. 2003;4(2):137–153. [PubMed]
  • Kuperminc GP, Leadbeater BJ, Blatt SJ. School social climate and individual differences in vulnerability to psychopathology among middle school students. Journal of School Psychology. 2001;39(2):141–159.
  • Kuperminc GP, Leadbeater BJ, Emmons C, Blatt SJ. Perceived school climate and difficulties in the social adjustment of middle school students. Applied Developmental Science. 1997;1:76–88.
  • Loukas A, Robinson S. Examining the moderating role of perceived school climate in early adolescent adjustment. Journal of Research on Adolescence. 2004;14(2):209–233.
  • McDonald JF, Moffitt RA. The use of tobit analysis. Review of Economics and Statistics. 1980;62:318–321.
  • McEvoy A, Welker R. Antisocial behavior, academic failure, and school climate: A critical review. Journal of Emotional & Behavioral Disorders. 2000;8(3)
  • Reinke WM, Herman KC. Creating school environments that deter antisocial behaviors in youth. Psychology in the Schools. 2002;39(5):549–559.
  • Roeser RW, Eccles JS. Adolescents’ perceptions of middle school: Relation to longitudinal changes in academic and psychological adjustment. Journal of Research on Adolescence. 1998;8(1):123–158.
  • Roeser RW, Eccles JS, Sameroff AJ. School as a context of early adolescents’ academic and social–emotional development: A summary of research findings. Elementary School Journal. 2000;100(5):443–471.
  • Stewart EA. School social bonds, school climate, and school misbehavior: A multilevel analysis. Justice Quarterly. 2003;20(3):575–604.
  • Sugai G, Horner R. Discipline and behavioral support: Practices, pitfalls, and promises. Effective School Practices. 1999;17:10–22.
  • Tobin J. Estimation of relationships for limited dependent variables. Econometrica. 1958;26(1):24–36.
  • Trzesniewski KH, Moffitt TE, Caspi A, Taylor A, Maughan B. Revisiting the association between reading achievement and antisocial behavior: New evidence of an environmental explanation from a twin study. Child Development. 2006;77(1):72–88. [PubMed]
  • Weishew NL, Peng S. Variables predicting students’problem behaviors. Journal of Educational Research. 1993;87:5–17.